An Algorithm of Solving Interlink Overlapping Ambiguity and Combinatorial Ambiguity and Compound Ambiguity in Chinese Word Segmentation

摘要

This article chiefly studies the characteristics of overlapping ambiguity and combinatorial ambiguity, defines interlink overlapping ambiguity field and compound ambiguity field, and points out the Bug on solving interlink overlapping ambiguity, combinatorial ambiguity and compound ambiguity field in Chinese word segmentation of the present several kinds of mature Chinese automatic word segmentation system, for example: the Institute of Computing Technology Chinese Lexical Analysis System(ICTCLAS), the HLSplitWord intelligent system, Institute of Computational Linguistics (ICL) Peking University online Chinese word segmentation system. In order to solve combinatorial ambiguity, this article corrects the semantic relevancy through contextual information based on Hownet, redefines the fee about the candidate word, and amends the algorithm of maximum probability word segmentation, solves successfully the questions about overlapping ambiguity, combinatorial ambiguity, interlink overlapping ambiguity and compound ambiguity in Chinese word segmentation. Finally, this article points out the calculation method of the semantic relevancy about the two words must be improved in a sentence.