A novel approach to improve the Record Linkage process

摘要

Organizations around the world lose trillions of dollars due to poor data quality problems. In the last years, the awareness about the importance of data quality made the stakeholders invest a lot of money in order to improve the quality of the stored data. One of the main processes in data quality field is the Record Linkage process (RL). Record linkage is the process of identifying the tuples that refer to the same real world entity. Without blocking, the RL process can end up by billions of comparisons when dealing with large datasets. Blocking reduces the number of comparisons by dividing the data into blocks in a way that only the records in the same block will be compared to each other. In this paper, we propose a novel approach in the record linkage field based on the K-Modes algorithm as a blocking step, K-Modes propose a major advantage because the algorithm deals directly with the categorical data. The Obtained results showed that our proposition stands as a powerful approach in the record linkage field where it outperforms most of the approaches that exist in the literature.