Semantic three-stream network for social relation recognition

摘要

In this paper, we propose a semantic three-stream network (STN) for social relation recognition, which learns discriminative features from facial images directly by exploiting semantic information effectively. Specifically, we employ a semantic augmentation structure to extract enriched semantic features from original face images, where a Siamese network is used to extract features from a pair of face images. We concatenate features of three streams for social relation recognition. Unlike most existing relation recognition methods, our method explicitly uses semantic information to discover the social relation. The proposed semantic augmentation structure can be easily embedded into the off-the-shell deep neural network, which leads to a powerful and flexible semantic augmentation network. Experimental results show that our proposed method outperforms the state-of-the-arts. (C) 2019 Elsevier B.V. All rights reserved.