Improved Boosting Algorithm with Adaptive Filtration

摘要

AdaBoost is known as an effective method to improve the performance of base classifiers both theoretically and empirically. However, previous studies have shown that AdaBoost is always prone to overfitting especially in noisy case. In addition, most current works on Boosting assume that the loss function is fixed and therefore do not take the distinction between noisy case and noise-free case into consideration. In this paper, an improved Boosting algorithm with adaptive filtration is proposed. A filtering algorithm is designed firstly based on Hoeffding Inequality to identify mislabeled or atypical samples. By introducing the filtering algorithm, we manage to modify the loss function such that influences of mislabeled or atypical samples are penalized. Experiments performed on eight different UCI data sets show that the new Boosting algorithm almost always obtains considerably better classification accuracy than AdaBoost. Furthermore, experiments on data with artificially controlled noise indicate that the new Boosting algorithm is more robust to noise than AdaBoost.