Multi-agent based individual web spider system

摘要

With the increasing web information and structure diversity of multi-element heterogeneous information, traditional information collection is limited in application. Individual information collection service and intelligence of web information collection become the focus. This paper adopts the technology of individual web spider and designs a multi-agent - based individual web spider system. The individual web spider model is the emphasis of this paper, besides system architecture, work mechanism and system components. The system consists of delivering agent, processing agent, analyzing agent, cooperation agent and adopts cooperation agent to resolve cooperation problems among agents. This system has been proved to be very effective through many experiments in different conditions.