ABSTRACT Convolutional neural networks (CNNs) have played a predominant role in the field of remote sensing over the last few years. As a significant branch of remote sensing image processing, pan-sharpening technique is to produce a high-resolution multi-spectral (HRMS) image based on a low-resolution multi-spectral (LRMS) image and a high-resolution (HR) panchromatic (PAN) image. Benefiting from the inherently powerful representation ability, deep-learning-based methods have also achieved promising and favourable performance in pan-sharpening community. However, these methods don’t take advantage of the gradient characteristic which contains abundant structure information to guide the pan-sharpening process, failing to achieve the desired spatial preservation. In this paper, we propose a gradient-guided pyramidal convolution residual network with interactive connections (GGPCRN) to relieve the above issue. Specifically, besides the indispensable reconstruction branch, an auxiliary gradient branch providing additional structure information is built to guide the recovery process. Moreover, we introduce pyramidal convolution containing a series of filters with varying depth and size into our network to capture different scales of details for better performance. To further enhance the guidance of gradient maps, two measures are taken. On the one hand, interactive connections are proposed to transfer the mutual effect between the reconstruction branch and gradient branch. On the other hand, we incorporate a mild gradient loss to force a second-order restraint on the pan-sharpened images, making the network concentrate more on structure preservation. Both reduced-resolution and full-resolution experiments suggest that our GGPCRN performs favourably against other methods in terms of quantitative evaluations and visual improvements.
展开▼
