State-of-the-art of 3D facial reconstruction methods for face recognition based on a single 2D training image per person

摘要

3D facial reconstruction systems attempt to reconstruct 3D facial models of individuals from their 2D photographic images or video sequences. Currently published face recognition systems, which exhibit well-known deficiencies, are largely based on 2D facial images, although 3D image capture systems can better encapsulate the 3D geometry of the human face. Accordingly, face recognition research is gradually shifting from the legacy 2D domain to the more sophisticated 2D to 3D or 2D/3D hybrid domain. Currently there exist four methods for 3D facial reconstruction. These are: Stochastic Newton Optimization method (SNO) [Blanz, V., Vetter, T., 1999. A morphable model for the synthesis of 3D faces. In: Proc. 26th Annu. Conf. on Computer Graphics and Interactive Techniques, SIGGRAPH. pp. 187-194; Blanz, V., Vetter, T., 2003. Face recognition based on fitting a 3D morphable model. IEEE Trans. Pattern Anal. Machine Intell. 25(9), 1063-1074; Blanz, V., 2001. Automatische Rekonstruction der Dreidimensionalen Form von Gesichtern aus einem Einzelbild. Ph.D. Thesis, Universitat Tubingen, Germany] inverse compositional image alignment algorithm (iCIA) [Romdhani, S., Vetter, T., 2003. Efficient, robust and accurate fitting of a 3D morphable model. In: IEEE Int. Conf. on Computer Vision, vol. 2, no. 1. pp. 59-66], linear shape and texture fitting algorithm (LiST) [ Romdhani, S., Blanz, V., Vetter, T., 2002. Face identification by fitting a 3D morphable model using linear shape and texture error functions. In: Proc. ECCV, vol. 4. pp. 3-19), and shape alignment and interpolation method correction (SAIMC) [Jiang, D., Hu, Y., Yan, S., Zhang, L., Zhang, H., Gao, W., 2005. Efficient 3D reconstruction for face recognition. Pattern Recogn. 38(6), 787-798]. The first three, SNO, ICIA + 3DMM, and LiST can be classified as "analysis-by-synthesis" techniques and SAIMC can be separately classified as a "3D supported 2D model". In this paper, we introduce, discuss and analyze the difference between these two frameworks. We begin by presenting the 3D morphable model (3DMM; Blanz and Vetter, 1999), which forms the foundation of all four of the reconstruction techniques described here. This is followed by a review of the basic "analysis-by-synthesis" framework and a comparison of the three methods that employ this approach. We next review the "3D supported 2D model" framework and introduce the SAIMC method, comparing it to the other three. The characteristics of all four methods are summarized in a table that should facilitate further research on this topic.