Contradiction Detection between Opinions From a Big Data perspective

摘要

This paper offers a solution to the problem of detecting contradictions among opinions on the same topic. The opinions are extracted from a large number of unstructured documents and stored in a structured format. Due to the increase in data available for analysis, we focus on providing a storage/retrieval and analysis solution suitable for managing large quantities of data while maintaining the speed and reliability present in smaller scale systems. Our approach consists in building a distributed system able to scale horizontally with the increase in input data without any significant performance decay. We represent opinions in a tuple based structured model, more suitable for retrieval and analysis. This approach allows us to formalize an algorithm for detecting contradictions between opinion tuples. Furthermore, we present a method for improving the recall of the system by using synonyms for the opinion target to expand the set of possible contradicting opinions. Our main focus is to optimize the structure of the opinion tuple to provide the best retrieval time and to allow for a simple, structured approach for detecting contradictions.