该文针对多尺度变换融合图像中普遍存在的需要依据先验知识选取滤波器,导致融合效果存在不确定性的问题,提出了基于深度堆叠卷积神经网络的融合方法.首先,分别以高斯拉普拉斯滤波器和高斯滤波器为首层网络的初始卷积核,将源图像分解为高频和低频图像序列;其次,基于He K方法初始化其余层卷积核,获得与源图像尺寸相同的高频和低频重构图像各一幅,并将二者合成源图像的近似图像;再以源图像和近似图像像素值之差的平方和的均值为误差函数,进行反向传播训练形成基本神经单元;之后,将多个基本单元堆叠起来利用end-to-end 的方式调整整个网络得到深度堆叠神经网络.然后,利用该堆叠网络分别分解测试图像对,得到各自的高频和低频图像,再基于局部方差取大和区域匹配度合并的规则分别融合高频和低频图像,并将高频融合图像和低频融合图像放回最后一层网络,得到最终的融合图像.实验结果表明;与基于双树复小波变换(Dual-Tree Complex Wavelet Transform,DTCWT)、非下采样轮廓波变换(Non-Subsampled Contourlet Transform,NSCT)和非下采样剪切波变换(Non-Subsampled Shearlet Transform,NSST)的融合结果相比,用高斯拉普拉斯滤波器和高斯滤波器初始化的深度堆叠卷积神经网络融合效果主观效果好,客观指标最优个数为NSCT的3.3倍,运行时间为NSCT的30.3％和NSST的11.6％.%When images are fused with multi-scale transform (MST),filter selection depends on prior knowledge,which causes the uncertainty problem of fused result.In order to address this problem,we propose an image fusion method based on the deep stack convolutional neural network.The method includes two parts.(1) One part is training deep stack convolutional neural network (DSCNN).It can be realized through three steps.Firstly,the initial convolution kernels are constructed with Gauss-Laplace filter and Gaussian filter in the first layer of network,respectively.So that the images can be decomposed into high-and low frequency image sequences by placing them to the input layer of the network.Secondly,the rest of convolution kernels of the network are initialized using the He K method to acquire reconstructed images of high-and low frequency in the same size as the source image,and then synthesize the two into an approximation image of the source image.Then,the basic neural unit is formed by back propagation training on the basis of error function,equal to the average of squares of the pixel value difference between the source image and approximate image,and the training data consists of different types of images,aiming to ensure the generalization ability of the network.Thirdly,the DSCNN is formed by stacking several basic units in the end-to-end manner and then training it use same error function and train data with basic neural unit.(2) The other part is fusing images with the DSCNN.It also contains three phases.Firstly,Using the DSCNN decompose test images so as to obtain high-and low frequency images,respectively.Secondly,the high frequency images are fused by choosing the maximum local variance and the low frequency images are fused in the principle of weighted average based on matched-degree.Thirdly,the final fused image is achieved by placing the two fused images to the last layer of DSCNN.The method is inspired by the MST method,which decomposes test images into high-and low frequency images and then fuse respectively.But we use the deep convolutional neural network to learn the decomposing and reconstructing filters,so that it is not necessary to manually define the number and types of filters,or to select the number of scale and directions.The method can adaptively decompose and reconstruct test images,which much less reduce depends on the fusion algorithm on prior knowledge.Multiple experimental results show that the DSCNN which is initialized using Gauss-Laplace filter and Gaussian filter is in subjective evaluation better than dual-tree complex wavelet transform (DTCWT),non-subsampled contourlet transform (NSCT) and non-subsampled shearlet transform (NSST).We has adopted standard deviation (SD),entropy (E),mutual information (MI),contrast (C),average gradient (AG),spatial frequency (SF) and uniform image quality indicators (UIQI)which are popular objective evaluation metrics for image fusion.The results show that the optimal objective indicators are 3.3 times that of NSCT,and the run time was 30.3％ of NSCT’s and 11.6％ of NSST's.
展开▼
