A Focused Crawler for Web Feature Service and Web Map Service Discovering

摘要

Due to globalization and the technological advances of the last decades, a large amount of data is created every day, especially on the Web. Web Services are one of the main artifacts created on the Web; they can provide access to sources of data of many sizes and types. In this work, we approach the challenge of designing and evaluating a Web Crawler to find two OGC (Open Geospatial Consortium) web services standards: WFS (Web Feature Service) and WMS (Web Map Service). Commercial search engines, e.g., Google and Bing, are indubitably doing a useful job as general-purpose search engines. However, some applications require domain-specific search engines and Web crawlers to find detailed information on the Web. Therefore, this work presents Spati-Harvest, a Web crawler that focuses on the task of discovering WFSes and WMSes. SpatiHarvest is a combination of some of the most advanced techniques found in the literature. Through experiments, we demonstrate that it is capable of finding more WFSes and WMSes, with less effort, when compared to the state of the art.