Gaussian process regression with heteroscedastic noises - A machine-learning predictive variance approach

摘要

Gaussian process regression (GPR) is one of the most important data analytic tools in modelling processes. It has attracted increasing interest in chemical engineering applications due to its superior performance in dealing with complex modelling problems such as high dimensional and nonlinear data. However, traditional GPR has the main limitation in that it considers an independent identically distributed (i.i.d.) noise at every sample point. Modern chemical processes typically have a more complex data structure and noise properties. The assumption of i.i.d. noise is not realistic. Thus, there is a growing interest in solving a heteroscedastic noise problem that does not satisfy the i.i.d. condition. The most common heteroscedastic noise is the noise with varying variance. This paper proposes a novel machine learning variance prediction method to solve the heteroskedastic GPR problem. By considering not only the input-dependent noise variance but also the input-output dependent noise variance, a regression model based on support vector regression (SVR) and extreme learning machine (ELM) method is proposed for both noise variance prediction and smoothing. Compared with the existing weighted Gaussian process regression (W-GPR) of the literature, the proposed method not only expands the use of W-GPR but also improves the prediction performance of heteroscedastic GPR models. Finally, the proposed algorithm is verified by two numerical examples and tested in a real polyester polymerization process. The results all demonstrate the effectiveness of the proposed approach. (C) 2020 Institution of Chemical Engineers. Published by Elsevier B.V. All rights reserved.