基于N-gram信息的中文文档分类研究

摘要

Traditional document classifiers are based on keywords in the documents, which need dictionaries support and efficient segmentation procedures. This paper explores the problem of utilizing N-gram information to categorize Chinese documents so that the classifiers can shake off the burden of large dictionaries and complex segmentation procedures,and subsequently be domain and time independent. Such a Chinese documents categorization system is implemented with kNN classification method. Experimental results show that it can achieve comparable performance to other classifiers of the same type.%传统文档分类系统都是基于文档的词属性，分类过程需要庞大的词典支持和复杂的切词处理。本文研究基于N-gram信息的中文文档分类，使中文文档分类系统摆脱对词典和切词处理的依赖，从而实现中文文档分类的领域无关性和时间无关性。利用kNN分类方法，实现了一个基于N-gram信息的中文文档分类系统。测试结果表明该文档分类系统具有和其它同类文档分类系统相当的性能。