Early Integration Testing for Entity Reconciliation in the Context of Heterogeneous Data Sources

Abstract

Entity reconciliation (ER) aims to combine data from different sources for a unified vision. The management of large volumes of data has given rise to significant challenges to the ER problem due to facts such as data becoming more unstructured, unclean, and incomplete or the existence of many datasets that store information about the same topic. Testing the applications that implement the ER problem is crucial to ensure both the correctness of the reconciliation process and the quality of the reconciled data. This paper presents an approach based on model-driven engineering that allows the creation of test models for the early integration testing of ER applications, contributing in three main aspects: the description of the elements of the proposed framework, the definition of the testing model, and the validation of the proposal through two real-world case studies. This validation verifies that the early integration testing of the ER application is capable of detecting a series of deficiencies, which a priori are not known and that will help to improve the final result that the ER application offers.