Pose-Invariant Facial Expression Recognition Based on 3D Face Morphable Model and Domain Adversarial Learning

摘要

Pose is one of the most important factors affecting performance of face related recognition algorithms including facial expression recognition (FER). Traditionally, non-frontal FER is conducted by either performing face formalization or designing separate models for different poses. Different from those methods, we propose a one-stage FER approach by training a pose invariant deep convolutional network (DCNN) with the following novelties: First, we introduce the 3D face morphable model to reconstruct high fidelity 3D faces for data augmentation which increases the pose variety without losing expression information. Second, we employ domain adversarial learning to eliminate the influence of domain difference between real 2D face images and 3D synthetic face images at feature level, which realizes a one-stage deep FER approach that is robust to different face poses. Third, the proposed approach provides a solution for cross-domain problems involving data from different sources, which can be applied to other face related recognition problems. The method is validated using three FER datasets FER2013, multi-PIE and BU-3DFE; and it outperforms the current state-of-the-art methods.