Using MDE for the Reconciliation of Entities in Large Data Sources

Abstract

With the birth of the Big and Open Data new business opportunities have been created in which the handling of a great quantity of information and its quality, positions any company with clear advantage with respect to its competitors. However, when working with large volumes of data, we find a very important problem called entity reconciliation. This problem is based on the difficulty of identifying the same real-world entity in different data sources. In this article, a proposal based on the Model-Driven Engineering (MDE) paradigm of and virtual graphs technology is presented to solve this problem. MDE is chosen for the ease that it provides to make our solution scalable to any domain and virtual (or implicit) graphs, for the advantages of using any type of algorithm based on graphs to find a solution to a specific problem and the necessity of not storing all the information we have in the same structure because of its size. Finally, a real case study is presented in which this solution is applied, concretely, for the management of large volumes of data of the historical heritage of the Andalusia region of Spain.