Constructing Comparable Corpora with Universal Similarity Measure

摘要

The invention describes a system and method for creating a comparable corpus by obtaining a set of source documents containing text, constructing language-independent semantic structures for at least one sentence of each of the texts in the source documents; determining universal similarity measures for groups of the source documents by comparing the constructed language-independent semantic structures of the texts in the source documents; identifying sets of similar documents based on the determined universal similarity measures for the groups of the source documents; and creating the comparable corpus based on the identified sets of similar documents.