Towards Multi-Objective Test Case generation for Variability-Intensive Systems

Abstract

Testing variability-intensive systems is a challenge due to the potentially huge number of derivable configurations. To alleviate this problem, many test case selection and prioritization techniques have been proposed with the aim of reducing the number of configurations to be tested and increasing their effectiveness. However, we found that these approaches do not exploit all available information since they are mainly driven by functional information such as the feature coverage. Furthermore, most of these works are focused on a single-objective perspective (e.g. features coverage), which could not reflect the real scenarios where several goals need to be met (e.g. features coverage and code changes coverage). In this context, we identify an important challenge, to take advantage of all available system information to guide the generation of test cases. As a first step towards a solution, we propose to study all this information with special emphasis on non-functional properties and address the test case generation as a multi-objective problem. Also, we describe some open issues to be explored that we hope have an important impact on future evaluations.