Aggregation of Information Resources on the Invisible Web

摘要

There are huge numbers of valuable information resources resided on Invisible Web. However, it is hard to use for us. In this paper we propose a system called NewsReaper that is capable of making Invisible Web to be visible, especially the huge number of real-time information, which update frequently and are time-sensitive. NewsReaper makes use of information extraction, text classification, full text index, RSS technologies to aggregate Invisible Web information resource. In first, this paper analyzes the reasons why it is invisible and four types of Invisible Web. At the same time it summarizes the characteristics of Invisible Web. On this basis, the system architecture of NewsReaper will to be introduced. In order to verify this system, there is a test for campus recruitment information compared with general-purpose search engines. Finally, the weaknesses of this system and some further works are discussed.