Evaluation of generality for multi-language of word segmentation method using inductive learning

摘要

We have proposed a method that segments a text into words by using inductive learning, and confirmed the performance to Japanese and Chinese word segmentation by experiments respectively. The method uses only the surface information of characters, so that it is independent on any specific language and a general method. In this paper, we do experiments of Japanese text and Chinese text with same algorithm simultaneously to demonstrate the generality of the method. The results of experiment show that the method is possible to be used to word segmentation of general non-segmented language.