Morphing billboards: an image-based appearance model for hand tracking

摘要

We present a novel approach to image-based rendering of articulated objects which gives reliable estimates of the object's shape and shading in new, previously unseen poses. The main idea is to combine a kinematic 3D model with a novel extension to billboard rendering in combination with a computationally lightweight axis-aligned morphing technique. The result is both visually appealing and accurate with respect to resolving texture, shading and elastic deformation of the articulated object. We show experimentally that our lightweight morphing technique allows the formulation of bias-free pixel-wise objective functions that improve upon static texturing. Using synthetic, ground-truth labelled data, we demonstrate that the proposed function is highly robust against image blur, occlusion, cluttered or skin-coloured background and exhibits significantly less local optima than the state of the art. In addition, we apply our method to real camera images and show several examples constructed from synthetic and real hand images.