Incorporating Posterior Estimates into AdaBoosti

摘要

Although boosting methods [9, 23] for creating compositions of weak hypotheses are among thebest methods of machine learning developed so far [4], they are known to degrade performance in case ofnoisy data and overlapping classes. In this paper we consider binary classification and propose a reduction ofoverlapping classes' classification problem to a deterministic problem. We also devise a new upper generali-zation bound for weighted averages of weak hypotheses, which uses posterior estimates for training objectsand is based on proposed reduction. If we are given accurate posterior estimates, this bound is lower thanexisting bound by Schapire et al. [22]. We design an AdaBoost-like algorithm which optimizes proposed gen-eralization bound and show that when incorporated with good posterior estimates it performs better than thestandard AdaBoost on real-world data sets.