Panoramic stereo reconstruction using non-SVP optics

摘要

Omni-directional sensors are useful in obtaining a 360° field of view with a single lens camera. Omni-directional stereo imaging systems can be constructed using a single camera and a mirror consisting of two concentric, radially symmetric lobes. If the central camera-mirror axis is vertical, a stereo image containing imagery from all azimuth directions from two viewpoints can be captured within one image. A vertically oriented catadioptric optical system that employs a mirror with a radial profile other than parabolic or hyperbolic cannot maintain the straightness of non-vertical lines. However, non-parabolic or hyperbolic mirror profiles are desirable for resolution distribution, sensor size, and cost. These mirror shapes, including spherical mirrors, are said to be non-central or non-single view-point (SVP); without a virtual perspective point, they pose new feature extraction challenges. The projection of straight lines in the world do not map to straight lines on the pixel array for non-SVP panoramic images, nor in the general case map to any conventional conic. A method for processing the imagery from a panoramic non-SVP catadioptric stereo sensor to reconstruct a 3D model of polyhedral objects with horizontal and vertical line edges is introduced. Horizontal line segments are extracted using the Panoramic Hough Transform and vertical line segments are recognized as straight radial lines. These segments, and closed shapes they form, are matched between the two lobe views to estimate position in three-dimensions. A novel Hough transform technique is described, and practical considerations for its successful use are explored. Four general weaknesses with Hough transform techniques in general are identified, and solutions to address them in each case are given. Details of a complete system using the Panoramic Hough Transform are given to demonstrate where this transform must be supplemented with other methods for robust operation. Issues of reconstruction accuracy are addressed, and experimental results with synthetic and real images are presented.