Accuracy Crawler: An Accurate Crawler for Deep Web Data Extraction

摘要

With the daily amalgamation in the size of data available on internet, the size of deep web is also continuously growing. The large size of the deep web in comparison with the surface web makes it very difficult to locate various deep web resources. In addition with harvesting the large size of the deep web content, classification of this content accurately is one of the major challenge. We propose a framework, namely Accurate Crawler, for accurately harvesting deep web content. Our crawler provides accurate classification of the deep web content by avoiding visiting a large number of pages. Accurate Crawler ranks sites based on the similarity of the content available, resulting in more accuracy in terms of site classification and extraction of deep web content. Accuracy Crawler has an excavating mechanism and an advanced relevance calculation mechanism to harvest relevant links by link ranking. Our experimental results on a set of representative domains show the accuracy of our proposed crawler framework that is higher than other crawler.