An Effective Method to Extract Web Content Information

摘要

To simplify the operation of web text content extraction and improve the accuracy of that, a newextraction method based on text-punctuation distribution and tag features (TPDT) is proposed. Combining the distribution of text-punctuation and tag features. Calculating the text-punctuation density in different text blocks and get the maximum continuoussum of density to extracting the best text content from web pages.The method effectively solves the problem of noisy information filtering and text content extraction without the training and manual processing. Experimental results on web pages randomly selected from different portalwebsites show that the TPDT method has good applicability on various news pages.