Automatic Linking of Similar Texts Across Languages

摘要

Cross-lingual Document Similarity calculation (CLDS) is useful for the navigation of large multilingual document collections and for clustering and classifying documents together independently of their language. We achieve CLDS by mapping documents onto the multilingual EUROVOC thesaurus, which will soon exist in 21 languages, and by representing each document in this multilingual vector space so that the similarity of texts written in different languages can be calculated. An evaluation showed that the system successfully identifies document translations in a large text collection. To adapt the method to the analysis of large multilingual news collections, we combined the mapping with cluster analysis and named entity recognition.