AN INFORMATION-THEORETIC APPROACH TO THE PRE-PRUNING OF CLASSIFICATION RULES

摘要

The automatic induction of classification rules from examples is an important technique used in data mining. One of the problems encountered is the overfitting of rules to training data. In some cases this can lead to an excessively large number of rules, many of which have very little predictive value for unseen dala. This paper is concerned with the reduction of overfitting. It inlroduces a technique known as J-pruning, based on the J-measure, an information theoretic means of quantifying the information conlenl of a rule and applies this to two rule induction methods: one where the rules are generated via the intermediate representation of a decision tree and one where rules are generated directly from examples.