Using evolution strategy for cooperative focused crawling on semantic web

摘要

Conventional focused crawling systems have difficulties on contextual information retrieval in semantic web environment. In order to deal with these problems, we propose a cooperative crawler platform based on evolution strategy to build semantic structure (i.e., local ontologies) of web spaces. Mainly, multiple crawlers can discover semantic instances (i.e., ontology fragments) from annotated resources in a web space, and a centralized meta-crawler can carry out incremental aggregation of the semantic instances sent by the multiple crawlers. To do this, we exploit similarity-based ontology matching algorithm for computing semantic fitness of a population, i.e., summation of all possible semantic similarities between the semantic instances. As a result, we could efficiently obtain the best mapping condition (i.e., maximizing the semantic fitness) of the estimated semantic structures. We have shown two significant contributions of this paper; (1) reconciling semantic conflicts between multiple crawlers, and (2) adapting to evolving semantic structures of web spaces over time.