Taggers for Unknown Words using Decision Tree and Lazy Learning

摘要

This paper describes methods of tagging unknown words using decision tree induction and lazy learning as post processes of morphological analysis. Unknown words are words which cannot be looked up in a dictionary of a natural language processing system. These are often named entities such as the name of a person, organization, or location. And these words play an important role in the application of information extraction. In this paper, we apply automated taggers that learn from training data which is written by hand. The algorithms of the taggers are a decision tree generated by C4.5 and a lazy learning algorithm. Attributes used for accurately tagging are notation of unknown words and the part of speech of neighboring words, and tag sets we classify these words are noun and some named entities. The experimental result of evaluating this method shows correct answers of 65% using decision tree tagger, and 70% using lazy learning tagger in open tests. We compared the decision tree tagger and the lazy learning tagger to show the advantages and disadvantages of each.