Object detection and localization in 3D environment by fusing raw fisheye image and attitude data

摘要

In robotic systems, the fisheye camera can provide a large field of view (FOV). Usually, the traditional restoring algorithms are needed, which are computational heavy and will introduce noise into original data, since the fisheye images are distorted. In this paper, we propose a framework to detect objects from the raw fisheye images without restoration, then locate objects in the real world coordinate by fusing attitude information. A deep neural network architecture based on the MobileNet and feature pyramid structure is designed to detect targets directly on the fisheye raw images. Then, the target can be located based on the fisheye visual model and the attitude of the camera, Compared to traditional approaches, this approach has advantages in computational efficiency and accuracy. This approach is validated by experiments with a fisheye camera and an onboard computer on a micro-aerial vehicle (MAV). (C) 2019 Elsevier Inc. All rights reserved.