Synthesis of 3D face structure with arbitrary expressions using difference vectors of expressions for frontal image

摘要

To realize natural human interface, there are many studies to synthesize facial expressions using computer graphics. In this paper, our goal is to reconstruct a 3D structure of a face with arbitrary expressions (expression-B) using 3D structure with expression-A, two images with expression-A, and domain knowledge of expressions. First, from input data, two projection matrices, which project the 3D structure to two input images, are estimated. Second, deformation from expression-A to expression-B is calculated as the difference vector of corresponding points in the images with expression-A and -B, where the images with expression-A and -B are from the image sample database. By using the difference vectors, similar property of face, symmetrical property of face and geometrical constraint, the images with expression-B are estimated. If two projection matrices and two facial images with expression-B are given, 3D structure with expression-B can be reconstructed. Experimental results show that 3D structure with expression could be reconstructed with fairly good performance.