A system of automatic keywords extraction and related data collection for Japanese text

摘要

Rapid progress of information services has been increasing the amount of text information which users recieve on Internet, teletext broadcasting and so on. A system extracting inportant part automatically is useful for users to get their interestinformation easily in such a case. Some methods in the field of natural language processing can extract important keywords or phrases from text information by using word dictionaries. however, these methods can hardiy deal with unexpected words such asnew proper nouns.On the other hand, the methods based on word occurrence are useful for dealing with unexpected words and they have high applicability for many kinds of text information because they use no dictionary. However, the results are insufficent as a summary insome cases.In this paper, we describe our newly developed KEIFIS (Keyword Extracting and Information Filtering System). KEIFIS extracts important keywords which represent major topics from a large amount of Japanese text information and collects related data with no dictionary.KEIFES has the following features:(1) It automatically extracts important keyowrds and combines some of them to give the users major topics.(2) It retrieves and collects information according to the topic specified by the users, and informs them of its arrival in real time.KEIFIS employs fuzzy information processing in computing the similarity between words. The similarity is also used to define a relationship between the specified topic and newly provided information.We applied KEIFIS to news programs on the teletextbroadcasting in Japanese, and made sure that KEIFIS was capable of extracting the important keywords which represent major topics appropriately.