Predicting Human Grasp Locations on Cup Handles by Using Deep Neural Networks to Infer Heat Signatures from Depth Data

摘要

In automated assisted living where a robot assists a human to interact with physical objects, an important challenge is for a robot to understand where humans are likely to grasp objects, so that the robot can present the object to a user in the most tenable configuration. In this paper, we present an approach that uses encoder-decoder convolutional neural networks (CNNs) to predict human grasp location on cup handles. The primary challenge addressed by our work is that object occlusion induced by the human hand prevents direct imaging of grasp location. Our approach uses the insight that once the object is released, the hand leaves a heat signature on the object surface due to the temperature differences between the human body and the ambient environment. Our CNNs learn a mapping between images obtained from traditional depth sensors as input and heat signatures of grasp locations imaged using a thermal camera as output. Given the depth image of a novel cup, our approach uses the trained network to predict the grasp probability distribution over the cup. Using a leave-one-cup-out approach, we obtain a mean absolute pixel-wise prediction error of 5.67 on 17 cups imaged from 7 orientations.