Noise level penalizing robust Gaussian process regression for NIR spectroscopy quantitative analysis

摘要

In Near-infrared (NIR) spectroscopy qualitative analysis, noise caused data quality problem has been a bottleneck to further enhance the prediction accuracy. Appropriate preprocessing methods can reduce the influence of noise; and robust models have higher tolerance for noise disturbance. However, these methods treat all the wavelengths equally. In fact, the spectra at different wavelengths may have highly different level of noise. This paper presents a new noise-level-penalizing robust Gaussian process (NLP-RGP) regression for NIR spectroscopy quantitative analysis. The novel noise level penalizing mechanism penalize the spectra features according to their noise level, i.e., encourage the model to prefer the less noisy features over high noisy features. Gaussian process (GP) is a nonparametric machine learning method based on kernel and Bayesian inference framework; with a noise model of heavy-tailed distribution, robust Gaussian process can handle the abnormal sample data better. Experiments were taken on the determination of the total soluble solids content of navel oranges based on their surface NIR spectra. The NLP-GP outperforms the robust Gaussian process model and least squares support vector machines (LS-SVM), the state of art method. Moreover, the NLP-RGP performs even better than the NLP-GP, achieving the best prediction accuracy among all the models. This demonstrates the effectiveness of noise level penalizing mechanism, and the noise level penalizing mechanism and robust mechanism of Gaussian process can be integrated together well.