Article
Multi-objective test case prioritization in highly configurable systems: A case study
Author/s | Parejo Maestre, José Antonio
Sánchez Jerez, Ana Belén Segura Rueda, Sergio Ruiz Cortés, Antonio López Herrejón, Roberto E. Egyed, Alexander |
Department | Universidad de Sevilla. Departamento de Lenguajes y Sistemas Informáticos |
Publication Date | 2016 |
Deposit Date | 2017-05-31 |
Published in |
|
Abstract | Test case prioritization schedules test cases for execution in an order that attempts to accelerate the detection of
faults. The order of test cases is determined by prioritization objectives such as covering code or ... Test case prioritization schedules test cases for execution in an order that attempts to accelerate the detection of faults. The order of test cases is determined by prioritization objectives such as covering code or critical components as rapidly as possible. The importance of this technique has been recognized in the context of Highly-Configurable Systems (HCSs), where the potentially huge number of configurations makes testing extremely challenging. However, current approaches for test case prioritization in HCSs suffer from two main limitations. First, the prioritization is usually driven by a single objective which neglects the potential benefits of combining multiple criteria to guide the detection of faults. Second, instead of using industry-strength case studies, evaluations are conducted using synthetic data, which provides no information about the effectiveness of different prioritization objectives. In this paper, we address both limitations by studying 63 combinations of up to three prioritization objectives in accelerating the detection of faults in the Drupal framework. Results show that non–functional properties such as the number of changes in the features are more effective than functional metrics extracted from the configuration model. Results also suggest that multi-objective prioritization typically results in faster fault detection than mono-objective prioritization. |
Project ID. | TIN2012-32273
TIN2015-70560-R P12-TIC- 1867 |
Citation | Parejo Maestre, J.A., Sánchez Jerez, A.B., Segura Rueda, S., Ruiz Cortés, A., López Herrejón, R.E. y Egyed, A. (2016). Multi-objective test case prioritization in highly configurable systems: A case study. Journal of Systems and Software, 122 (december 2016), 287-310. |
Files | Size | Format | View | Description |
---|---|---|---|---|
MultiObj-TestCase-Prio.pdf | 3.082Mb | [PDF] | View/ | |