基于网页结构相似度的Web 信息抽取

摘要

In this paper, we focus on Web Information Extraction by use of structural similarities.Combined with URL similarity, the Tree- Edit- Distance method is adapted to measure the similarity between web pages.The similarity is used for detecting changes between the pages as the criterion.By this way, the suitable extraction rules will be selected, otherwise,the target page will be fed to rule learner module.Extraction rules are induced automatically by machine learning.The introduction of the algorithm given extraction system the self-renewal capacity , which make it better adapt to the practical application for the extraction of heterogeneous resources.The feasibility and efficiency of the algorithm in the prototype system was able to verify.%本文重点探讨基于编辑距离的网页相似度算法在Web 抽取系统中的应用与实现.通过结合基于URL 及编辑距离的网页结构相似度的计算方法,抽取系统在抽取过程中能够检测网页结构的变化,从而主动做出判断,选择适应规则进行抽取或通过主动学习自动扩展规则库.结构相似度计算赋予系统感知网页结构变化的能力,系统通过主动自我更新与调整,能更好地适应面向实际应用的异构资源的获取.算法的可行性和效率在原型系统中得以验证.