Ensemble Validation: Selectivity has a Price, but Variety is Free

摘要

For an ensemble classifier that is composed of classifiers selected from a hypothesis set of classifiers, and that selects one of its constituent classifiers at random to use for each classification, we present ensemble error bounds consisting of the average of error bounds for the individual classifiers in the ensemble, a term that depends on the fraction of hypothesis classifiers selected for the ensemble, and a small constant term and multiplier. There is no penalty for using a richer hypothesis set, if the same fraction of the hypothesis classifiers are selected for the ensemble.