Identifying and Resolving Conflicts in Multi Source Data

摘要

While access to data is continually increasing, it is unclear whether access to reliable information is increasing at close to the same rate, if at all. Data inconsistencies are ubiquitous; as data arrives from multiple sources, analysts and other data consumers must determine what data to accept and what to reject. Today's data consumers need the ability to automatically detect when conflicts occur and examine these inconsistencies in context-using an understanding of temporal evolution to consider how different conflicts may be related, and how the underlying sources factor into the credibility of the information. In response to this need, we are developing inference algorithms that detect and resolve conflicting information in multi-source text data by placing the potentially conflicting information in context, to suggest resolutions and to understand the social connections behind different pieces of conflicting information. We embed our techniques in a system called REsolving Differences via Inference (RED1). Our approach uses statistical and graph-based characterizations of data to describe expected structures, and identify combinations of data that deviate from them. Using the expected structures, we then propose resolutions to the data conflicts.