Utilizing Genetic Algorithm and Online Resources for Vietnamese Text Categorization

摘要

The fundamental difference between English and Vietnamese, in which a sentence in Vietnamese may not have a unique segmentation with a unique meaning, has created great challenges for research in Vietnamese text categorization. Most of the related works for English and Western languages are not applicable for text categorization in Vietnamese due to this fundamental difference. The lack of training data also imposes many constraints on our capacity in exploiting different methods. There have been similar research efforts in Chinese and Japanese but most require large sets of training data which are particularly hard to find or compose in Vietnamese. This paper presents IGATEC (Internet and Genetics Algorithm-based Text Categorization), a novel text categorization approach that requires no training data or dictionary. The possibility of different text segmentations with different meaning is also taken in to account. This novel approach is a combination of Genetics Algorithm with statistical data extracted from the Internet. These data are easily extracted by any available internet search engine, and are stored for off-line used in both IGATEC's GA sentence segmentation engine and text classifier for speeding up. We achieved a notable accuracy of 97.1% in classifying news collected from Vietnamese online news with a short running time.