A Novel Feature Selection Method Based on Category Information Analysis for Class Prejudging in Text Classification

摘要

This paper presents a new feature selection algorithm with the category information analysis in text classification. The algorithm obscure or reduce the noises of text features by computing the feature contribution with word and document frequency and introducing variance mechanism to mine the latent category information. The algorithm is distinguished from others by providing a pre-fetching technique for classifier while it is compatible with efficient feature selection, which means that the classifier can actively prejudge the candidate class labels to unseen documents using the category information linked to features and classify them in the candidate class space to retrench time expenses. The experimental results on Chinese and English corpus show that the algorithm achieves a high performance. The F measure is 0.73 and 0.93 respectively and the run efficiency of classifier is improved greatly.