Fine-grained image inpainting with scale-enhanced generative adversarial network

摘要

With the emergence of Generative Adversarial Networks, great progress has been made in image inpainting. However, most existing methods can produce plausible results, but fail to generate finer textures and structures. This is mainly due to the fact that (1) the generation of finer content in the masked region of an image is not constrained enough during network training, and (2) many different alternative pixels are exist to fill in the masked regions, making it very difficult for the inpainting network to generate reasonable sharp edges. To address these issues, we propose a Scale Enhanced GAN (SE-GAN) model which combines the constraints of large- and small-scale receptive fields of our tailor-made discriminators to achieve fine-grained constraint on image details, a novel edge loss to further ensure the sharpness of the generated image. Experiments on multiple datasets including faces(CelebA-HQ), textures(DTD), buildings(Facade) and natural images(ImageNet, Places2) show that our approach can generate higher quality inpainting results with more details than previous methods. (c) 2021 Elsevier B.V. All rights reserved.