Using mobile multi-camera unit for real-time 3D motion estimation and map building of indoor environment

摘要

Real-time 3D map building and 3D motion estimation using only visual data are two challenging problems which have been intensively studied by the machine vision community in the past decade. In order to successfully build a 3D map, the accurate 3D motion estimation of the input sensor during the map building process is needed. Up to now, most of the attempts to improve the 3D motion estimation process have been concentrated on the software algorithms used. However, despite the use of sophisticated algorithms, accurate 3D motion information is still hindered by the limitation of the visual sensor used, e.g. a single camera with small field of view which suffers the motion ambiguity problem in the case of small movements, leading to inaccurate motion information and poor map quality.This thesis work proposes a new piece of multi-camera hardware to be used as a 3D visual sensing device for the real-time 3D motion estimation and 3D map building problems. Instead of focusing only on the software solution, this work takes an alternative approach to improve the motion estimation accuracy and robustness by means of a better hardware design. A multi-camera unit (MCU) which is aimed for high accuracy 3D motion detection is constructed. It consists of three pairs of stereo cameras which are put together as a compact, mobile hardware platform. This unique camera arrangement eliminates the motion ambiguity error found in single camera systems and so accurate motion estimation is obtained. The increased field of view by means of multiple cameras also enables a simple but accurate detection of 3D movement of the camera in real-time without any complex calculations. The accompanied algorithms which are needed for the real-time 3D motion estimation including the real-time feature detection and feature matching as well as outlier rejection schemes are also implemented for the MCU system. Moreover, the FastSLAM algorithm for real-time 3D localization and map building approach is implemented in order to maintain a consistent feature point map and the location and orientation of the MCU. As a result, the proposed 3D motion estimation and 3D map building using the MCU system gives a better performance compared to the conventional, single camera systems as confirmed by the simulation results and real world experiments. This is especially the case for 3D motion estimation performances, where the motion ambiguity error is being compensated in both rotation and translation cases. The probabilistic approach for 3D feature point map building shows a strong real-time performance and consistency with good accuracy. Finally, the proposed multi-camera hardware is used for a 3D photorealistic map building task where a high quality 3D model which correctly replicates the surrounding environment can be constructed in real-time.