Forum Data Extraction without Explicit Rules

摘要

Web forum data contributed by millions of users are the mixture of well-formed user information and free-format user-generated content. Though easy to read for users, forum data are difficult to be analyzed by computer systems because of various surrounding HTML tags. It is challenging to extract forum data from a large number of Web sites automatically since these sites may have different styles. In this paper, we propose an approach to extract user information and user-generated content from multiple forum sites by using both structural and textual characteristics of forums. A structural induction process and a term combination computation process are introduced to assure extraction accuracy and automation. Extensive experiments on real-life data sets show the effectiveness of our proposed method.