Spectral bounding: Strictly satisfying the 1-Lipschitz property for generative adversarial networks

摘要

Imposing the 1-Lipschitz constraint is a problem of key importance in the training of Generative Adversarial Networks (GANs), which has been proved to productively improve stability of GAN training. Although some interesting alternative methods have been proposed to enforce the 1-Lipschitz property, these existing approaches (e.g., weight clipping, gradient penalty (GP), and spectral normalization (SN)) are only partially successful. In this paper, we propose a novel method, which we refer to as spectral bounding (SB) to strictly enforce the 1-Lipschitz constraint. Our method adopts very cost-effective terms of both 1-norm and infinity-norm, and yet allows us to efficiently approximate the upper bound of spectral norms. In this way, our method provide important insights to the relationship between an alternative of strictly satisfying the Lipschitz property and explainable training stability improvements of GAN. Our proposed method thus significantly enhances the stability of GAN training and the quality of generated images. Extensive experiments are conducted, showing that the proposed method outperforms GP and SN on both CIFAR-10 and ILSVRC2015 (ImagetNet) dataset in terms of the standard inception score. (C) 2019 Elsevier Ltd. All rights reserved.