An Evaluation of the Methods to Convert Non-segmented 'kana' Strings to 'kanji-kana' Strings Using Markov Chain Models

摘要

Up to now, the method to convert non-segmented "kana" strings to "kanji-kana" strings by looking up all possible candidates of words in the dictionary and by using 2nd-order Markov chain model of "kanji-kana" characters, has been proposed. However, this method takes a lot of time to look up all possible candidates in the dictionary of large vocabulary (e.g. four hundred and thirty thousand words). This paper proposes a new method of "kana-to-kanji" conversion. This method is composed of two processing. One is the processing to find the provisional boundaries of "kana" words. The other is the processing to convert the "kana" word strings to "kanji-kana" words strings. The proposed method is evaluated by the experiments using statistical data for 1,677 issue of daily newspaper. From the results of the experiments, it is concluded that the proposed method is superior to the conventional method in accuracy rate of "kana-to-kanji" conversion and processing time.