Semi-supervised learning for facial expression-based emotion recognition in the continuous domain

摘要

Emotion recognition is a very important technique for effective interaction between human and artificial intelligence (AI) system. For a long time, facial expression-based methods have been actively studied, and they are showing high recognition performance thanks to powerful deep learning recently. On the other hand, the images of the datasets used in the conventional emotion recognition studies are usually short in length and often generated through intentional expression. Also, continuous domain annotation of emotional labels in dataset configuration requires high cost. In order to overcome such problems, this paper proposes an emotion recognition method based on semi-supervised learning that utilizes an appropriate amount of unlabeled dataset in parallel while minimizing the use of labeled dataset requiring high training cost. The proposed emotion recognition method is based on CNN-LSTM-based regressor for regressing arousal and valence in continuous domain. In addition, we present scenarios and design criteria in which semi-supervised learning can be effectively applied to emotion recognition tasks through experiments using well-known MAHNOB-HCI and AFEW-VA datasets.