A Graded Approach for the Efficient Resolution of Chinese Word Segmentation Ambiguities

摘要

This paper presents a new approach for the efficient resolution of Chinese segmentation ambiguities. The approach includes the discrimination between ambiguous and unambiguous words, the simple and complex ambiguity classification, and the handling methods for each kind of ambiguities. The basic idea is to solve as many as ambiguities at the lower levels, so that most ambiguities can be resolved at simple disambiguation phase. The approach has been implemented in our SLex system that currently supports the processing of a large-scale Chinese corpus. Some relevant design and implementation details that support this technique are also discussed.