Tibetan-Chinese Cross Language Text Similarity Calculation Based onLDA Topic Model

摘要

Topic model building is the basis and the most critical module of cross-language topic detection and tracking.Topic model also can be applied to cross-language text similarity calculation. It can improve the efficiency and the speedof calculation by reducing the texts’ dimensionality. In this paper, we use the LDA model in cross-language text similaritycomputation to obtain Tibetan-Chinese comparable corpora: (1) Extending Tibetan-Chinese dictionary by extractingTibetan-Chinese entities from Wikipedia. (2) Using topic model to make the texts mapped to the feature space of topics.(3) Calculating the similarity of two texts in different language according to the characteristics of the news text. Themethod for text similarity calculation based on LDA model reduces the dimensions of text space vector, and enhances theunderstanding of the text’s semantics. It also improves the speed and efficiency of calculation.