Topical Crawler based on multi-level vector space model and optimized hyperlink chosen strategy

摘要

In this study, through researching and analyzing the technology of Topical Crawler, we improve the critical algorithm, and present the Topic-Relevance judgment algorithm based on Multi-Level Vector Space Model and the Topical Search Strategy based on the content evaluation, which combine with the simple link structure analysis and link tags data analysis. Through the analysis of the experimental data, the algorithm and the strategy proposed in this research have higher accuracy and efficiency, which can improve the performance of the Traditional Topical Crawler greatly.