一种基于词聚类的文本特征描述方法

摘要

针对文本挖掘中存在的特征空间高维性问题,提出了一种基于词聚类的文本特征描述方法,旨在通过机器学习的方法挖掘词汇之间的语义关联,动态构造特定领域的概念词典,借助构造的概念来描述文本的特征,该方法不借助主题词典,先从训练语料中对词的共现情况进行分析,用词聚类(word clustering)生成由种子词(seedwords)表示的代表某一主题概念的词类,然后用种子词作为文本的特征项.实验表明,该方法不仅压缩了特征空间的维数,也克服了HowNet中概念信息的局限性,提高了文本分类的精确度.%Feature space has the high-dimensional problem in text mining. This paper presented a new description method of text feature based on word clustering. The purpose is to mine semantic association between words using machine learning, then to construct the concept dictionary in specific areas dynamically, finally to describe the text feature with the concept constructed. This method analyzes the co-occurrence of words in training corpus firstly, without using theme dictionary, then generates word cluster expressed in seed words which represents a concept of theme by word clustering, finally takes the seed words as text features. The experimental results indicate that this method not only reduces dimensionality of feature space but also overcomes the limitations of the concept in HowNet, and improve the performance of text categorization.