In-house Crowdsourcing-Based Entity Resolution: Dealing with Common Names

摘要

Entity Resolution (ER) is one of the techniques used to disambiguate the various manifestations of same object to improve search results in databases. Recently, Crowd sourcing has been utilized to improve entity resolution to gain positive impact when searching for particular information in a database. In this paper, we consider the domain of Customer Relationship Management (CRM) and utilize the approach of Crowd sourcing to enrich the process of achieving ER. Specifically our focus is to identify the right customer that has been manifested in various ways under a common name in a database using In-house Crowd sourcing-based Entity Resolution approach (ICER). The ICER takes the list of possible duplicates into consideration (which are pre-determined) and identifies the pair of record that has the maximum impact in achieving ER. Then, this pair is crowd sourced to Customer Service Representatives (CSRs) to have their input (labeling). ICER incorporates the principles of Human Intelligence Task (HIT) that aims to keep the questions asked to the CSR to a minimum. Two ICER approaches are proposed in this study based on probabilistic (modified approach of Whang et al) and active learning schemas. The applicability of the proposed ICER approaches and comparison of their results have been highlighted by using an example.