Increasing the performance and consistency of classification trees by using the accuracy criterion at the leaves

摘要

The traditional split criteria in tree induction (Gini, Entropy and others) do not minimize the number of misclassifications at each node, and hence cannot correctly estimate the parameters of a tree, even if the underlying model can be correctly modeled by a tree procedure. We examine this effect and show that the difference in accuracy can be as much as 15precent in the worst case. We prove that using the Gini criterion, trees unbounded in size may be grown in order to correctly estimate a model. We then give a procedure that is guaranteed to give finite trees and define a modification to the standard tree growing methodology that results in improvements in predictive accuracy from 1precent to 5precent on datasets from the UCI repository.