一种中文领域概念词自动提取方法研究

摘要

For statistical method lacks semantic information between words in domain concepts extraction, this paper presents a domain concept automatic extraction method, which combines semantic similarity and improved affinity propagation. The compound words are extracted by using mutual information, and then the log-likelihood is used to avoid the omission of low-frequency words, after that the synonyms between terms are identified by using HowNet and the cosine similarity. The improved affinity propagation algorithm is used to obtain the collection of domain concepts. The experimental results show that the method has higher accuracy, recall rate, and perplexity change ratio than the traditional method.%针对统计学方法在领域概念获取时缺少词语语义信息的问题，提出了一种结合语义相似度和改进近邻传播算法的领域概念自动获取方法。该方法通过互信息进行合成词提取，使用对数似然比避免对低频词的遗漏，利用HowNet和余弦相似度识别术语间同义词，采用改进的近邻传播算法获取领域概念集合。实验结果表明，该方法在准确率、召回率和困惑度变化率上比传统的方法都有较大提高。