Web-based classification using machine learning approaches.

摘要

This thesis describes an application to automate web page classification based on the Yahoo hierarchy. Machine learning approaches developed for learning on text data are used here on the hierarchical classification structure. The high number of features is reduced by taking into account the hierarchical structure and using feature selection based on the method known from IR (information retrieval). Documents are represented as feature-vectors that include the standard “bag of words” model as commonly used when learning on text data. Based on the hierarchical structure the problem is divided into different web sites, each representing one document on the categories included in the Yahoo hierarchy. The result of learning is a set of rules, each used to match the probability that a new example is a member of the corresponding category. Our classifier uses a voting technique to choose the category prediction that receives the highest combination score over each new page with n set of rules that match to the category to classify and predict that category. Thus we use only the most accurate of all applicable rules to classify a new page. (Abstract shortened by UMI.)