Anime Sketch Coloring with Swish-gated Residual U-net and Spectrally Normalized GAN

摘要

Anime sketch coloring is to fill various colorsinto the black-and-white anime sketches and finally obtainthe color anime images. Recently, anime sketch coloring hasbecome a new research hotspot in the field of deep learning. Inanime sketch coloring, generative adversarial networks (GANs)have been used to design appropriate coloring methods andachieved some results. However, the existing methods basedon GANs generally have low-quality coloring effects, such asunreasonable color mixing, poor color gradient effect. In thispaper, an efficient anime sketch coloring method using swishgated residual U-net (SGRU) and spectrally normalized GAN(SNGAN) has been proposed to solve the above problems.The proposed method is called spectrally normalized GANwith swish-gated residual U-net (SSN-GAN). In SSN-GAN,SGRU is used as the generator. SGRU is the U-net with theproposed swish layer and swish-gated residual blocks (SGBs).In SGRU, the proposed swish layer and swish-gated residualblocks (SGBs) effectively filter the information transmitted byeach level and improve the performance of the network. Theperceptual loss and the per-pixel loss are used to constitutethe final loss of SGRU. The discriminator of SSN-GAN usesspectral normalization as a stabilizer of training of GAN, andit is also used as the perceptual network for calculating theperceptual loss. SSN-GAN can automatically color the sketchwithout providing any coloring hints in advance and can beeasily end-to-end trained. Experimental results show that ourmethod performs better than other state-of-the-art coloringmethods, and can obtain colorful anime images with highervisual quality.