Numerical methods for accelerating the PCA of large data sets applied to hyperspectral imaging

摘要

Principal component analysis and regression (PCA, PCR) are widespread algorithms for the calibration of spectrometers and the evaluation of spectra. In many applications, however, there are huge amounts of calibration data, as it is common to hyperspectral imaging for instance. Such data sets consist often of several ten thousands of spectra measured at several hundred wavelength positions. Hence, a PCA of calibration sets that large is computational very time consuming - in particular the included singular value decomposition (SVD). Since this procedure takes several hours of computation time on conventional personal computers, its calculation is often not feasible. In this paper a straightforward acceleration of the PCA is presented, which is achieved by data preprocessing consisting of three steps: data compression based on a wavelet transformation, exclusion of redundant data, and by taking advantage of the matrix dimensions. Since the size of the calibration matrix determines the calculation time of the PCA, a reduction of its size enables the acceleration. Due to an appropriate data preprocessing, the PCA of the discussed examples could be accelerated by more than one order of magnitude. It is demonstrated by means of synthetically generated spectra as well as by experimental data that after preprocessing the PCA results in calibration models, which are comparable to the ones obtained by the conventional approach.