A Denoising Method for Light Field Imaging Sensor Based on Spatial-Angular Collaborative Encoding Network

摘要

Light field (LF) imaging sensors based on micro-lens array are susceptible to numerous noise pollution when collecting raw 4D LF data due to their own special structural design, which affects visual perception and subsequent applications such as depth estimation. Unfortunately, existing 2D image denoising methods are difficult to directly apply to 4D LF images. To this end, this paper proposes a new LF denoising method based on spatial-angular collaborative encoding network, considering the inherent 4D structure of LF image. Specifically, the convolutions in the spatial and angular branches are first constructed to extract specific 2D spatial and 2D angular features from noisy LF data. Then, a tailored spatial-angular collaborative encoder is designed to co-process spatial-angular features and improve the expression ability of features. After that, the spatial-angular feature fusion module is constructed to fuse the extracted features. Finally, the denoised LF image is reconstructed by a residual prediction module integrating attention mechanism. In particular, the proposed method simultaneously reconstructs all sub-aperture images of LF in one forward inference, so as to preserve the angular consistency of the denoised LF image. Extensive experimental results show that the proposed method outperforms the state-of-the-art methods in both subjective visual perception and objective quality evaluation. Furthermore, the proposed method preserves the parallax structure well, which is beneficial for subsequent LF applications.