Shape Tensors for Efficient and Learnable Indexing

摘要

Multi-point geometry: The geometry of 1 point in N images under perspective projection has been thoroughly investigated, identifying bilinear, trilinear, and quadrilinear relations between the projections of 1 point in 2, 3 and 4 frames respectively. The dual problem - the geometry of N points in 1 image - has been studied mostly in the context of object recognition, often assuming weak perspective or affine projection. We provide here a complete description of this problem. We employ a formalism in which multi-frame and multi-point geometries appear in symmetry: points and projections are interchangeable. We then derive bilinear equations for 6 points (dual to 4-frame geometry), trilinear equations for 7 points (dual to 3-frame geometry), and quadrilinear equations for 8 points (dual to the epipolar geometry). We show that the quadrilinear equations are dependent on the the bilinear and trilinear equations, and we show that adding more points will not generate any new equation.rnApplications to reconstruction and recognition: The new equations are used to design new algorithms for the reconstruction of shape from many frames, and for learning invariant relations for indexing into a data-base. We describe algorithms which require matching 6 (or more) corresponding points from at least 4 images, 7 (or more) points from at least 3 images, or 8 (or more) points from at least 2 images. Unlike previous approaches, the equations developed here lead to direct and linear solutions without going through the cameras' geometry. Our final linear shape computation uses all the available data - all points and all frames simultaneously: it uses a factorization of the matrix of invariant relations into 2 components of rank 4, a shape matrix and a coordinate-system matrix.