Deep multi-person kinship matching and recognition for family photos

摘要

In this paper, we propose a novel Deep Kinship Matching and Recognition (DKMR) framework for multi-person kinship matching and recognition, which is a complicated and challenging task with little previous literature. Compared with most existing kinship understanding methods that mainly work on matching kinship in pairwise face images, we target at recognizing the exact kinship in nuclear family photos consisting of multiple persons. The proposed DKMR framework contains three modules. Firstly, we design a deep kinship matching model (termed DKM-TRL) to predict kin-or-not scores by integrating the triple ranking loss into a Siamese CNN model. Secondly, we develop a deep kinship recognition model (named DKR-GA) to predict the exact kinship categories, in which gender and relative age attributes are utilized to learn more discriminative representations. Thirdly, based on the outputs of DKM-TRL and DKR-GA, we propose a reasoning conditional random field (R-CRF) model to infer the corresponding optimal family tree by exploiting the common kinship knowledge of a nuclear family. To evaluate the effectiveness of our DKMR framework, we conduct extensive experiments and the results show that it can gain superior performance on Group-Face dataset, TSKinFace dataset and FIW dataset over state-of-the-arts. (C) 2020 Elsevier Ltd. All rights reserved.