Preliminary Comparison of Techniques for Dealing with Imbalance in Software Defect Prediction

Abstract

Imbalanced data is a common problem in data mining when dealing with classi cation problems, where samples of a class vastly outnumber other classes. In this situation, many data mining algorithms generate poor models as they try to opti- mize the overall accuracy and perform badly in classes with very few samples. Software Engineering data in general and defect prediction datasets are not an exception and in this paper, we compare different approaches, namely sampling, cost-sensitive, ensemble and hybrid approaches to the prob- lem of defect prediction with different datasets preprocessed differently. We have used the well-known NASA datasets curated by Shepperd et al. There are differences in the re- sults depending on the characteristics of the dataset and the evaluation metrics, especially if duplicates and inconsisten- cies are removed as a preprocessing step.