基于词聚类的热点话题检测算法

摘要

对应用词聚类进行热点话题检测的算法进行了研究.通过将文档分词并用兼顾长短文章的规则进行特征抽取,将文档聚类空间转化为特征词聚类空间,采用基于互信息的词聚类算法产生热点话题类.以TDT5语料作为测试语料进行了热点话题召回率和话题类纯度评测,实验结果表明,采用基于词聚类的算法进行热点话题检测,热点话题的平均召回率达到83.8%,话题类的平均类纯度达到94.4%,检测出的热点话题类别易于理解.%A new algorithm for hot topic detection using words clustering is proposed. First documents space is converted to words space by word segmentation and extraction of key words considering the length of the document, then hot topic cluster is detected using words clustering algorithm based mutual information. Hot topic recall and the pure of the topic is measured on the TDT5 corpus. Experiments show that the average recall of hot topic detection reaches to 83.8％, the average pure of the topic cluster reaches to 94.4％; the detected hot topic is easily understandable.