基于子树匹配的文本相似度算法

摘要

To reduce the dimensionality of text vectors and improve the performance of semantic similarity measurement, an algorithm for texts similarity computation is proposed, which combines the advantages of the statistical methods and semantic dictionary. The texts are utilized to generate metadata feature vectors, so that it reduces the dimensionality of text vectors space. The algorithm for computing texts similarity is designed based on subtrees matching and the speed of computing texts similarity is improved. The accuracy of texts semantic similarity measurement is improved by utilizing the semantic matching of metadata feature vectors and subtrees. The synonyms widely existing in metadata are processed by the proposed method, and the semantic coverage ability for similarity computation of texts is also enhanced. The experimental results show that the proposed method is feasible and effective.%为降低文本向量维度，提高文本间语义相似度度量性能，综合利用统计方法与语义词典的优势，提出一种文本相似度算法。基于文本生成元数据特征向量，减少向量空间维度，设计基于子树匹配的文本相似度算法，借助子树加速对文本相似度的计算，并通过将文本元数据特征向量与子树进行相似度语义匹配，提高文本相似度计算时语义相似度度量的准确性。该算法考虑到对元数据中同义词的语义理解，加强文本之间相似度度量时的语义覆盖能力。实验结果表明文中所提出的方法是可行和有效的。