An industrial missing values processing method based on generating model

摘要

The issue of missing values (MVs) has been found widely in real-world datasets and obstructed the use of many statistical or machine learning algorithms for data analytics due to their incompetence in processing incomplete datasets. Most of the current MVs filling methods are applied to the datasets with certain specific types or low missing rate. This paper proposes a method of missing values processing based on the combination of denoising autoencoder (DAE) and generative adversarial networks (GAN), aiming at the missing completely at random (MCAR) datasets with high missing rate and noise interference in industrial scenes. We execute the training process on a discrete dataset with missing values, in order to ensure the generated dataset is completely similar to the feature distribution of the original dataset. We conduct our experiments for different dimensional datasets to prove the feasibility and efficiency of this method, including three public authority datasets and an industrial production monitoring dataset. The results compared with traditional missing values imputation methods have shown when the missing rate is higher than 30%, our method performs better in robustness and accuracy. (C) 2019 Published by Elsevier B.V.