基于自学习向量空间模型文本分类算法的研究与应用

摘要

In recent years, information processing turns more and more important for us to get useful information. Text categorization, the automated assigning of natural language texts to predefined categories based on their contents, is a task of increasing importance. This paper gives a research to several key techniques about text categorization, in-cluding vector space model, feature extraction, machine learning. At the last of this paper, a text categorization system based on VSM and ML is implemented, and it also can dispose text categorization at real time through kafka and storm, which is a flow calculation framework.%随着网络信息的迅猛发展，信息处理已经成为人们获取有用信息不可缺少的工具。文本自动分类是信息处理的重要研究方向。它是指在给定的分类体系下，根据文本的内容自动判别文本类别的过程。本文对文本分类中所涉及的关键技术，包括向量空间模型，特征提取，机器学习方法等进行了研究和探讨。最后，本文实现了一套基于自学习向量空间模型的文本分类系统，并基于kafka消息队列和storm流计算框架，实时地为文本进行分类。