一种实体描述短文本相似度计算方法

摘要

Short text description for Chinese entities has features of statistical sparsity,semantic discretization and irregular vocabulary. This research analyses the relationship between sememe network and word similarity in Hownet and presents a short text description similarity computation method that consists of semantic similarity part and short text classification part. In the semantic similarity part,the method weakens the influence of Hownet’s shallow sememes and balances weights of sememes. In the short text classification part,the method transforms short texts into sememe vectors and classifies them according to the distribution of sememes in certain fields. Take average results of those two parts to generate short text de-scription similarity. Effectiveness of the method is proved by task 1 of Baidu knowledge map analyzing competition.%中文实体描述短文本具有词语稀疏、语义离散、用词随意等特点。本文分析《知网》义原网络和词语相似度的关系，提出了短文本间语义相似度部分和短文本分类部分相结合的实体描述短文本间相似度计算方法。语义相似度部分分析《知网》义原网络和词语间相似度的关系，在计算词语间相似度和短文本间相似度的过程中弱化了浅层《知网》义原影响并均衡了义原权重，使义原相似度计算结果更加合理。短文本分类部分将短文本分解为义原向量，根据特定领域短文本的义原分布情况进行短文本分类。两部分结合得到实体描述短文本间相似度。本文方法的有效性在百度知识图谱数据分析竞赛任务1的测试结果中得到了证明。