Double Encoder Conditional GAN for Facial Expression Synthesis

摘要

Photorealistic facial expression synthesis from single face image is already a highly challenging research work, in part due to a paucity of labeled and paired facial expression samples. Most existing facial expression synthesis works attempt to learn the transformation between expression domains and thus would require the paired samples as well as the labeled query image. In this paper, we propose the Double Encoder Conditional GAN (DECGAN) for facial expression synthesis. Generative Adversarial Networks (GANs) have demonstrated to successfully approximate complex data distributions. And cGANs, which contain external information, can determine the specific relationship between images. This work inspires us to modify the structure of GAN, and use the target facial expression feature as a condition. In this work, we propose two encoders to encode the original expression and the target expression, respectively, to extract the latent vectors and conditional labels features of the real image. In the meantime, associative learning is used to associate unpaired original emoticons with target emoticons in the database and to share identities.