基于结构树的网页正文内容抽取方法

摘要

网页文本抽取是一种在互联网上运用广泛的数据挖掘技术,主要目的是把一个网页的主题内容抽取出来,为Web数据挖掘提供好的基础数据.基于网页树形结构进行改进,首先对网页进行分块,把每一块存储在树形结构当中,然后通过对所有块进行方差和阈值计算,选择出主题信息.该方法相比传统的基于正则表达式的抽取方法,具有简单、实用的特点.实验结果表明,该抽取方法准确率达到96％以上,有一定的实用价值.%Content extraction is a kind of data mining technology which is widely used in internet The main purpose is to extract the topic content and provide the data for Web Data Mining. To improve Web-based tree structure , first of all the Webpage divided into blocks , to each block of which is stored in the tree structure, then all the blocks of variance and threshold calculation, choose the topic information. In comparison with traditional methods based on regular expressions, this method is more simple and useful. Experimental results show that the extraction precision is higher than 96% , and the method has good value of practice.