A new end-to-end framework based on non-local network structure and spatial attention mechanism for image rain removal

摘要

Image rain removal is an important topic in the field of computer vision. In the rainy environment, the rain will seriously affect the quality of imaging, resulting in image deformation, blur, poor visibility, and other problems. So the outdoor vision system cannot accurately detect object, monitor, and other works. Therefore, how to effectively eliminate the rain-weather interference to imaging has a very important practical value. In the absence of the time series information between frame and frame, the bottleneck problem of image rain removal technology is how to effectively remove multi-density rain-fringes while preserving the detailed structure information of the image background. To solve the above problems, a new image rain removal algorithm based on non-local network structure and spatial attention mechanism is proposed in this paper. Firstly, the position relation information between different pixels is obtained by the non-local operation to obtain the global image representation. Secondly, the spatial attention mechanism is used to recalibrate the global information in the spatial dimension. In other words, the nonlinear modeling is carried out on the channel dimension to gather similar features and useful information. Finally, deconvolution and long-distance residual connection are used to restore the size of the rain-removing image layer by layer. The analysis and experimental results show that the proposed algorithm in this paper is effective in removing rain marks, effectively solves the practical difficulties of removing rain stripes with different rain densities, and preserves the details and edge information of the image well.