同主题词短文本分类算法中BTM的应用与改进

摘要

In order to solve the problem of large-scale short-text corpus topic model parameter K, the FBTM model is proposed to reduce the sampling complexity from O (K) to O (1). Aiming at the short spelling of short text and the weak description ability, this paper proposes a short text classification algorithm with biterm with the same topic and FBTM. Firstly, we use FBTM to model the text, and extend the same topic biterm in a sliding window as feature in the original text. Then, we use the FBTM topic distribution as another part of the text feature. The results show that this method has significantly improved the classification performance of Weibo corpus.%为解决大规模短文本语料库主题模型参数K较大导致求解慢的问题,本文提出FBTM模型,将BTM中单个词对采样复杂度由O(K)降低O(1).针对短文本词语稀疏、描述能力弱的特点,提出一种结合同主题词对与FBTM的短文本分类算法,首先使用FBTM进行主题建模,将一段滑动窗口内的同主题词对作为特征扩充到原文本中,然后使用FBTM主题分布作为另一部分文本特征.对特征扩展后的Weibo语料库进行分类实验,结果显示该方法显著提高了分类性能.