Dropout for Ensemble Machine Learning Classifiers

摘要

Machine learning often uses ensemble classifiers, such as random forest or gradient boosting tree classifiers to solve problems. One issue with such classifiers is that they may be prone to data overfitting. This can cause the classifier to perform relatively worse when dealing with data outside of a training set. One technique to avoiding overfitting is using random dropout on decision trees in the ensemble classifier (e.g. drop three percent of all decision trees to create a final classifier). However, random dropout can be improved upon. Penalty based dropout can assess the performance of individual trees using a validation data set (which may be separate from the training set). Instead of using random dropout, some of the worst performing trees can be dropped instead, leading to better overall performance.