BETTY: Benchmarking and Testing on the Automated Analysis of Feature Models

Abstract

The automated analysis of feature models is a flourishing research topic that has called the attention of both researchers and practitioners during the last two decades. During this time, the number of tools and techniques enabling the analysis of feature models has increased and also their complexity. in this scenario, the lack of specific testing mechanisms to assess the correctness and good performance of analysis tools is becoming a major obstacle hindering the development of tools and affecting their quality and reliability. in this paper, we present BeTTy, a framework for BEnchmarking and TesTing on the analY sis of feature models. Among other features, BeTTy enables the automated detection of faults in feature model analysis tools. Also, it supports the generation of motivating test data to evaluate the performance of analysis tools in both average and pessimistic cases. Part of the functionality of the framework is provided through a web-based interface facilitating the random generation of both classic and attributed feature models.