Real-time Recovery of Non-rigid Face from Monocular Video

摘要

In this paper, we address the problem of estimating 3D pose and shape of non-rigid face from monocular video. The contributions of this paper are as follows. Firstly, we develop an efficient approach named LK-ASIM to track the facial features robustly by integrating the Lucas-Kanade optical tracking algorithm with active shape model face alignment algorithm. Secondly, the deformable 3D model is automatically built from a sequence of 2D LK-ASM tracking results by means of the non-rigid structure from motion algorithm. Finally, in order to obtain the 3D pose and shape of non-rigid facet we propose a nonlinear optimization method based on bundle adjustment to fit the deformable model. Experiments demonstrate the effectiveness of our proposed scheme in the real-time recovery of pose and shape of faces from monocular video.