International Speech Communication Association Distinguished Lecture: Principles and Design of a System for Academic Information Retrieval based on Human-Machine Dialogue

摘要

With the rapid progress of computer technology and world-wide development of information networks, a vast amount of information is now being generated, published, and stored at a number of sites distributed all over the world. Such an affluence of information, however, is useless or may even become harmful unless one has a means for rapidly retrieving the information that it truly necessary and appropriate. Conventional systems for information retrieval, however, are not always easy to use for inexperienced users, and are neither efficient nor accurate. In many cases, it is difficult for the user to identify and express his/her intention precisely, and it is difficult also for the system to infer the user's intention correctly. These difficulties can be alleviated by introducing spoken dialogue between the user and the system. Furthermore, in conventional systems using keywords, the accuracy of retrieval is reduced by the existence of synonymy, polysemy and homonymy, as well as of unknown words. Still another shortcoming of conventional systems is the lack of ability for properly estimating the degree of relevance of a document to the user's query, as well as the lack of a proper viewpoint on the cost/performance of retrieval. This talk describes the outcome of a successful Japanese national project conducted under the "Research-for-the-Future" program and led by the speaker as the principal investigator. The system is based on the following three original principles: (a) Dialogue Management based on both User and System Modeling (by introducing a novel type of interacting automaton), (b) Use of "Key Concepts" in information retrieval (including processing of polysemy, homonymy, and unknown words), and (c) Optimization of Retrieval Performance through Relevance Score Estimation (by introducing a measure of relevance of search results based on users' judgments. The advantages of these novel principles have been demonstrated by a pilot system.