一种融合词语位置特征的Lucene相似度评分算法

摘要

相似度评分算法是Lucene引擎中的核心部分之一。对Lucene内部的相似度评分算法进行研究分析后，针对Lucene只关心查询词出现的频率，而不关心它们所在的位置这一缺陷提出了一种改进的算法。改进的算法将词语位置关系特征融合到Lucene原始相似度评分算法中。在TREC数据集上的实验结果表明：改进后的算法与Lucene原始算法相比，在MAP和P@n指标上都有一定程度的提高。%The scoring algorithm of similarity is one of the core parts in Lucene. After the analysing and researching on the default scoring algorithm of Lucene similarity, this paper proposes an improved algorithm aimed at the deficiency of the Lucene’s default algorithm which only considers the frequencies rather than the position of query terms occurrence. The improved algorithm combines the feature of the terms’position relationship with Lucene’s default scoring algorithm of similarity. The experiment on the TREC dataset shows that, the improved algorithm increases the value of evaluation metric MAP and P@n to a certain extent.