Empirical software product line engineering: A systematic literature review

Abstract

Context: The adoption of Software Product Line Engineering (SPLE) is usually only based on its theoretical benefits instead of empirical evidences. In fact, there is no work that synthesizes the empirical studies on SPLE. This makes it difficult for researchers to base their contributions on previous works validated with an empirical strategy. Objective: The objective of this work is to discover and summarize the studies that have used empirical evidences in SPLE limited to those ones with the intervention of humans. This will allow evaluating the quality and to know the scope of these studies over time. Doing so, research opportunities can arise Methods: A systematic literature review was conducted. The scope of the work focuses on those studies in which there is human intervention and were published between 2000 and 2018. We considered peer-reviewed papers from journals and top software engineering conferences. Results: Out of a total of 1880 studies in the initial set, a total of 62 primary studies were selected after applying a series of inclusion and exclusion criteria. We found that, approximately 56% of the studies used the empirical case study strategy while the rest used experimental strategies. Around 86% of the case studies were performed in an industrial environment showing the penetration of SPLE in industry. Conclusion: The interest of empirical studies has been growing since 2008. Around 95.16% of the studies address aspects related to domain engineering while application engineering received less attention. Most of the experiments and case study evaluated showed an acceptable level of quality. The first study found dates from 2005 and since then, the interest in the empirical SPLE has increased.