Disturbing Neighbors Ensembles for Linear SVM

摘要

Ensembles need their base classifiers do not always agree for any prediction (diverse base classifiers). Disturbing Neighbors (DN) is a method for improving the diversity of the base classifiers of any ensemble algorithm. DN builds for each base classifier a set of extra features based on a 1 -Nearest Neighbors (1-NN) output. These 1-NN are built using a small subset of randomly selected instances from the training dataset. DN has already been proved successfully on unstable base classifiers (i.e. decision trees). This paper presents an experimental validation on 62 UCI datasets for standard ensemble methods using Support Vector Machines (SVM) with a linear kernel as base classifiers. SVMs are very stable, so it is hard to increase their diversity when they belong to an ensemble. However, experiments will show that DN usually improves ensemble accuracy and base classifiers diversity.