FREQUENCY INFORMATION EQUIPPED WORD SET GENERATION METHOD, PROGRAM, PROGRAM STORAGE MEDIUM, FREQUENCY INFORMATION EQUIPPED WORD SET GENERATION DEVICE, TEXT INDEX WORD PRODUCTION DEVICE, FULL TEXT RETRIEVAL DEVICE AND TEXT CLASSIFICATION DEVICE

摘要

PPROBLEM TO BE SOLVED: To extract a word having a possibility of identification to a language having a lot of ambiguity in the word identification such as a compound word as fully as possible, and to calculate the frequency information thereof. PSOLUTION: This frequency information-equipped word set generation device 1 includes: a morpheme lattice generation part 10 generating a morpheme lattice expressed as a graph wherein an inputted text is connected from the beginning of the text to the end of the text by a morpheme; a morpheme/word importance level calculation part 20 calculating a morpheme appearance probability about each morpheme included in the morpheme lattice, and calculating an importance level of each morpheme correspondingly to the morpheme appearance probability; and a frequency information equipped word set calculation part 30 calculating the frequency information about each word included in the morpheme lattice on the basis of the importance level of the morpheme, and generating a frequency information equipped word set based on a pair of the word and the frequency information. PCOPYRIGHT: (C)2006,JPO&NCIPI