Research of automatic Chinese word segmentation

摘要

Automatic Chinese word segmentation is the fundamental task of Chinese information processing. At present ambiguous phrase segmentation and proper name recognition are two obstacles to the performances of Chinese word segmentation systems. We apply a corpus-based method to extract various language phenomena from real texts, and combine a statistical model with rules in Chinese word segmentation, which has increased the precision of segmentation by improving ambiguous phrase segmentation and unknown word recognition, and finally, we describe a Chinese word segmentation system developed by Shanxi University.