Fully convolutional network-based infrared and visible image fusion

摘要

This study proposes a novel fusion framework for infrared and visual images based on a full convolutional network (FCN) in the local non-subsampled shearlet transform (LNSST) domain. First, the LNSST is used as a multi-scale analysis tool to decompose the source images into low-frequency and high-frequency sub-images. Second, the coefficients of the high-frequency sub-images are fed into the FCN to obtain the weight map, and then the average gradient (AVG) is used as the fusion rule to fuse the high-frequency sub-images while the low-frequency coefficients are fused by local energy fusion strategy. Finally, the inverse of the LNSST is applied to obtain the final fused image. The experimental results showed that the proposed fusion framework performed better than other typical fusion methods in both visual quality and objective assessment.