基于WMD距离与近邻传播的新闻评论聚类

摘要

随着新闻网站的快速发展,网络新闻和评论数据激增,给人们带来了大量有价值的信息.新闻让人们了解发生在国内外的时事,而评论则体现了人们对事件的观点和看法,这对舆情分析和新闻评论推荐等应用很重要.然而,新闻评论数据又多又杂,而且通常比较简短,因此难以快速直观地从中发现评论者的关注点所在.为此,该文提出一种面向新闻评论的聚类方法EWMD-AP,用以自动挖掘社会大众对事件的关注点.该方法利用强化了权重向量的Word Mover's Distance(WMD)计算评论之间的距离,进而用Affinity Propagation(AP)对评论进行聚类,从杂乱的新闻评论中得到关注点簇及其代表性评论.特别地,该文提出利用强化权重向量替代传统WMD中的词频权重向量.而强化权重由三部分组成,包括结合词性特征与文本表达特征的词重要度系数、新闻正文作为评论背景的去背景化系数和TFIDF系数.在24个新闻评论数据集上的对比实验表明,EWMD-AP相比Kmeans和Mean Shift等传统聚类算法以及Density Peaks等当前最新算法都具有更好的新闻评论聚类效果.%With the rapid development of news websites,the news comments increase sharply,which are very impor-tant to public opinion analysis and news comments recommendation.This paper proposes a news comments cluste-ring method,called EWMD-AP,to automatically mine the focuses of the public on the news.This method employs Word Mover's Distance (WMD) with enhanced weight vectors to calculate the distances between news comments.It also adopts Affinity Propagation (AP) to cluster comments,and finally obtains the clusters and their representative comments corresponding to the focuses of the public.Particularly,this paper proposes to replace the traditional word frequency based weight vectors in WMD with enhanced weight vectors,which consist of three components :the importance coefficient of words,the de-contextualization coefficient,and the traditional TFIDF coefficient.Ex-perimental results on 24 news comments datasets demonstrate that EWMD-AP performs much better than both tra-ditional clustering methods (e.g.Kmeans,Mean Shift,etc) and the state-of-the-art ones (e.g.Density Peaks, etc).