The interaction of sampling ratio and modelling method in prediction of binary target with rare target class

摘要

In many practical predictive data mining problems with a binary target, one of the targetudclasses is rare. In such a situation it is common practice to decrease the ratio of common toudrare class cases in the training set by under-sampling the common class. The relationshipudbetween the ratio of common to rare class cases in the training set and model performanceudwas investigated empirically on three artificial and three real-world data sets. The resultsudindicated that a flexible modelling method without regularisation benefits in both mean andudvariance of performance from a larger ratio when evaluated on a criterion sensitive toudoverfitting, and benefits in mean but not variance of performance when evaluated on audcriterion less sensitive to overfitting. For an inflexible modelling method and a flexibleudmethod with regularisation, the effects of a larger ratio were less consistent. In noudcircumstances, however, was a larger ratio found to be detrimental to model performance,udhowever measured.