BeTTy: Benchmarking and Testing on the Automated Analysis of Feature Models

Abstract

The automated analysis of feature models is a ourishing research topic that has called the attention of both researchers and practitioners during the last two decades. During this time, the number of tools and techniques enabling the analysis of feature models has increased and also their complexity. In this scenario, the lack of speci c testing mechanisms to assess the correctness and good performance of analysis tools is becoming a major obstacle hindering the development of tools and a ecting their quality and reliability. In this pa-per, we present BeTTy, a framework for BEnchmarking and T esT ing on the analY sis of feature models. Among other features, BeTTy enables the automated detection of faults in feature model analysis tools. Also, it supports the gen-eration of motivating test data to evaluate the performance of analysis tools in both average and pessimistic cases. Part of the functionality of the framework is provided through a web-based interface facilitating the random generation of both classic and attributed feature models.