Large-scale documents reduction based on domain ontology and E2LSH

摘要

Large-scale documents reduction plays a critical role in document management organizing and document mining, etc, and the research is concentrated on two special aspects: the construction of document representation model and index optimization of feature space for similarity search. While the semantic gap and curse of dimensionality are still two open and tough issues. Motivated by this, in the paper, we propose a novel method based on domain ontology and E2LSH (Exact Euclidean Locality-Sensitive Hashing). Firstly, we build an improved model based on domain ontology, called Semantic Vector Space Model (SVSM), to reveal the latent semantic relationships among document feature terms besides syntax information. The SVSM shortens the semantic gap of traditional VSM and reduces feature dimension. Then in view of the complexity of searching space for the similarity computation among documents pairs, we introduce E2LSH to build indexes of feature space, optimizing the searching space and overcoming the curse of dimensionality. Experimental validation has been conducted using realistic documents, and experimental results indicate the rationality and effectiveness of our method