A Web warehouse system for semi-automatically gathering and managing online news

摘要

In this paper we propose a Web warehouse system that gathers and manages online news in a semi-automatic fashion, serving as intermediate information repository for a given user community. We describe its architecture and an ontology-based, focused crawler for automatically collecting relevant news documents. We further discuss the problem of efficient management of the hit frequency profile for all visited news stories and propose a randomized data structure, ABF-Aging Bloom Filter, to cope with this problem. We demonstrate that the proposed system can save a good deal of Web traffic and online time when individual users try to search and retrieve the relevant online news.