Research on link blocks recognition of web pages

摘要

The link block is a typical type of block structure of web pages; it is an important research object in the fields of web data mining. Firstly, block and block tree are proposed as the basic concepts of subsequent explorations, and then an approach of building block trees is put forward. Secondly, four rules for link block discrimination and two indicators for recognition results evaluation are put forward based on the concept of block. Finally, a strategy named forward algorithm for discovery of link block (FAD) is proposed and a corresponding experiment with different parameters is performed to verify the strategy. The results show that the FAD can be flexible to achieve recognition of link blocks under different granularity conditions. Concepts and approaches presented in this paper have a good prospect in the fields of web data processing such as advertising block recognition and web content extraction.