Monocular template-based 3D surface reconstruction: Convex inextensible and nonconvex isometric methods

摘要

we study the 3D reconstruction of an isometric surface from point correspondences between a template and a single input image. The template shows the surface flat and fronto-parallel. We propose three new methods. The first two use a convex relaxation of isometry to inextensibility. They are formulated as Second Order Cone Programs (SOCP). The first proposed method is point-wise (it reconstructs only the input point correspondences) while the second proposed method uses a smooth and continuous surface model, based on Free-Form Deformations (FFD). The third proposed method uses the 'true' nonconvex isometric constraint and the same continuous surface model. It is formulated with Nonlinear Least-Squares and can thus be solved with the efficient Levenberg-Marquardt minimization method. The proposed approaches may be combined in a single pipeline whereby one of the convex approximations is used to initialize the nonconvex method. Our contributions solve two important limitations of current state of the art: our convex methods are the first ones to handle noise in both the template and image points, and our non-convex method is the first one to use 'true' isometric constraints. Our experimental results on simulated and real data show that our convex point-wise method and our nonconvex method outperform respectively current initialization and refinement methods in 3D reconstructed surface accuracy.