一种基于知识工程的DeepWeb信息抽取方法

摘要

The information contained in DeepWeb is more and more huge with great value. But due to the factors that DeepWeb informa-tion is highly heterogeneous, autonomous, dynamic and incomplete, and the design style, page structure, display contents of DeepWeb theme website are different,and the use of JavaScript technology is widespread,the traditional extraction technology can’ t be effectively automated integration of high quality information contained in DeepWeb resources. Presents a DeepWeb extraction method based on knowledge engineering. The page mode,HTML structure and visual features of DeepWeb are analyzed and integrated. Applies HTML DOM tree parsing algorithm to match the template accorded with page mode,HTML structure and object information source by automatic or semi-automatic way,locating the information in DeepWeb to obtain the free text,structured and semi-structured data. Using a large number of site data with nested structure as data source,the effectiveness of extraction method is verified.%DeepWeb中蕴含的信息越发庞大并且价值可观。但是由于DeepWeb信息的高度异构性、自主性、动态性以及不完整性,DeepWeb主题性网站的设计风格、页面结构、显示内容的不同,JavaScript技术的广泛使用等因素,使传统的抽取技术无法有效自动化集成蕴含在DeepWeb中的高质量信息资源。提出一种基于KBE(知识工程)的DeepWeb信息抽取方法。通过对目标DeepWeb的页面模式、页面HTML结构、页面视觉信息等进行分析、整合,利用HTML DOM ( Document Object Model)树解析算法,自动或半自动方式匹配出符合页面模式、页面HTML结构以及目标信息源等元素的模板,来对Deep-Web中的信息进行定位,从而得到页面中的自由文本,结构化和半结构化数据。实验以大量嵌套结构的网站数据作为数据来源,验证了抽取方法的有效性。