Real-Time Landing Spot Detection and Pose Estimation on Thermal Images Using Convolutional Neural Networks

摘要

This paper presents a robust, accurate and real-time approach to detect landing spot position and orientation information using deep convolutional neural networks and image processing technique on thermal images. The proposed novel algorithm pipeline consists of two steps: ledge detection and orientation information extraction. The extracted pose information of the landing spot from thermal images could be used to facilitate autonomous operations of unmanned aerial vehicles (UAVs) in both of day and night time. In order to land on the narrow and long ledge, UAV requires accurate orientation information of the ledge. Moreover, the method is scale and rotation invariant and also robust to occlusion in certain special and unexpected situations. Our algorithm runs at 20 frames per second on NVIDIA GTX 1080Ti GPU with the real flight thermal image dataset captured by T-Lion UAV developed by Temasek Laboratories@NUS.