Spectral Super-Resolution Using Hybrid 2D-3D Structure Tensor Attention Networks with Camera Spectral Sensitivity Prior

摘要

With the development of deep convolutional neural networks (CNNs), spectral super-resolution (SSR) has obtained a significant improvement, which aims to recover the hyperspectral image (HSI) from a single RGB. However, the existing mapping algorithms lack of utilization of the camera spectral sensitivity (CSS) and only focus on wider or deeper architecture design, neglecting to explore the feature correlations of intermediate layers, thus preventing the representational ability of CNNs. In our paper, a novel hybrid 2D-3D structure tensor attention networks (HSTAN) with CSS prior is proposed for SSR. In specific, a structure tensor attention (STA) embedded in the residual block is invented to extract the salient high-frequency spatial details for adequate spatial feature expression. Furthermore, the CSS is firstly exploited as a prior to avoid its influence of SSR quality, based on which the reconstructed RGB can be calculated naturally through the super-resolved HSI, then the final loss incorporates the discrepancies of RGB and the HSI as a finer constraint. Experimental results demonstrate the superiority of our proposed algorithm.