SmarTest: A Test Case Prioritization Tool for Drupal

Abstract

Test case prioritization techniques aim to identify the optimal ordering of tests to accelerate the detection of faults. The importance of these techniques has been recognized in the context of Software Product Lines (SPLs), where the potentially huge number of products makes testing extremely challenging. We found that the open source Drupal framework shares most of the principles and challenges of SPL development and it can be considered a real-world example of family of products. In a previous work, we represented the Drupal configuration space as a feature model and we collected extra functional information about its features from open repositories. Part of this data proved to be a good indicator of faults propensity in Drupal features. Thus, they become valuable assets to prioritize tests in individual Drupal products. In this paper, we present SmarTest, a test prioritization tool for accelerating the detection of faults in Drupal. SmarTest has been developed as an extension of the Drupal core testing system. SmarTest supports the prioritization of tests providing faster feedback and letting testers begin correcting critical faults earlier. Different test prioritization criteria can be selected in SmarTest, such as prioritization based on the number of commits made in the code, or based on the tests that failed in last executions. A customizable dashboard with significant system information to guide the testing is also provided by SmarTest at run-time. This work represents an interesting application of SPL-inspired testing techniques to real-world software systems, which could be applicable to other open-source SPLs.