基于语义信息的中文短信文本相似度研究

摘要

This paper analyzes The semantic information of characteristic words in Chinese SMS text based on the traditional model TF-ID, and proposes a Chinese SMS text similarity measurement method. It preprocesses the SMS text, calculates TF-1DF values of each word, and selects high TF-IDF value of words as keywords. With word vector similarity of the Vector Space Model(VSM), combined with the word similarity weighted, it gives two SMS text similarity computing method. Experimental results show that the F-measure of this method is better than TF-IDF and word semantic similarity algorithm.%在传统TF-IDF模型基础上分析中文短信文本中特征词的语义信息,提出一种中文短信文本相似度度量方法.对短信文本进行预处理,计算各词语的TF-IDF值,并选择TF-IDF值较高的词作为特征词,借助向量空间模型的词语向量相似度,结合词语相似度加权,给出2篇短信文本相似度的计算方法.实验结果表明,该方法在F-度量值上优于TF-IDF算法及词语语义相似度算法.