An algorithm for selecting Chinese features based on TF-NIDF weight

摘要

This article discusses the problem of selecting Chinese features based on TF-IDF weight in text categorization. TF-IDF weight is commonly used in text categorization for its simplexes. However, it can not express the relationship between a feature appearance frequency in one class and appearance frequency in other classes. To solve the problem, we designed TF-NIDF weighting method to express the relationship and computer feature weight. We also incorporated the weight into Naïve Bayesian classifier and tested it on Chinese text data. Experiments showed that Naïve Bayesian classifier with features selection based on TF-NIDF weight have a higher categorization precision than Naïve Bayesian classifier with features selection based on traditional TF-IDF weight.