面向不规则列表的网页数据抽取技术的研究

摘要

Data extracted from list page can be used for further data mining and data integration,etc.This paper studied how to improve the accuracy and adaptability of automatic extraction of list pages.Based on the existing algorithms of mining multi-ple data regions and data records,it improved the algorithm of identifing data records against the diversity of the organization of data records,and improved the accuracy of identifing data records.As for the irregularities of data records,it considered the content enclosed in nodes based on the algothrim of tree matching and improved its accuracy.Based on the comparion between tag trees which represented data records,it used the algorithm of tree alignment to generate an extraction model that contained all the data fileds.And the model would be used for extracion of all the data reocrds.The results show that the improved algo-rithm effectively improve the accuracy and adaptability when extracting data from list pages.%抽取列表页中的列表数据可以用于进一步的数据挖掘以及数据集成等系统。针对怎样提高自动抽取列表页数据的准确率和适应性进行了研究。在研究已有的多数据区域挖掘算法和数据记录识别算法的基础上，针对列表页数据记录组织方式的多样性改进了数据记录识别算法，提高了识别数据记录的准确率。而对于数据记录之间的不规则性问题，在已有的标签树匹配算法的基础上加入了对节点内容的考虑，提高了两棵标签树匹配的准确率。根据构成数据记录的标签树之间的匹配结果，再采用部分树对齐算法生成一个数据记录的最大匹配结构，进而用于抽取出所有数据记录。实验结果表明，提出的改进算法有效提高了自动抽取列表页数据的准确率和适应性。