One of the most important aspects in data preprocessing for data raining concerns the handling and imputation of missing values. While differences in the performance of varying state-of-the-art algorithms on the same dataset remain usually rather small, the quality of missing value handling can have dramatic consequences and is often crucial for the success of the following model building. This paper explores the consequences of two major missing value replacement strategies (replace-with-mean and multivariate regression) for the performance of classification models: By using a complete real-world dataset for a binary classification problem (churn in financial services), the hit rates of different data mining algorithms are benchmarked for the case of no missing values being present. Then, different missing value patterns (MCAR, MAR and IM) are simulated by deleting predictor values from the training samples following those patterns. After this, the two imputation strategies (replace with mean and regression) are used to recreate complete training datasets, in order to build classification models on them. Finally, the hit rates of the models are determined on (the original complete, not imputed) hold-out test sets and the performances of the models are compared. It is clearly shown, that the regression strategy outperforms by far the simpler replace-with-mean imputation by introducing much less artificial bias in the data and thus enabling better models to be built. The results underline the performance advantages of more complex and time-consuming multivariate imputation schemes over the straightforward replace-with-mean techniques unfortunately implemented in many commercial data mining packages.
展开▼
