Entity Matching Using Different Level Similarity for Different Attributes

摘要

Entity matching (EM) recognizes records in one or different databases that represent the same entity in real-world. It is an essential part of data cleaning and data integration. Most existing studies determine whether two records match by calculating the similarity of the corresponding attribute values in the two records. Then series of similarity metrics are proposed. But according to our investigation, we found that no one has considered combining semantic level similarity with string level similarity. In some data tables, some attributes values are long text, such as product descriptions, and they are more semantically similar in different records that refer to a same entity. Other numeric or noun attributes, such as price and person name, are more suitable for string level similarity calculations. Therefore, we propose a model to calculate the similarity of the two types of attributes using modules of different similarity levels, and assign them different weights. Learning by labeled data, we get a model that can effectively solve the entity matching task.