基于多模态深度学习的RGB-D物体识别

摘要

针对现有RGB-D物体识别方法存在图像特征学习不全面、类间相似的物体识别精度不高等问题,联合稀疏自编码(sparse auto-encoder,SAE)及递归神经网络(recursive neural networks,RNNs)提出多模态稀疏自编码递归神经网络(multi-modal sparse auto-encoder and recursive neural networks,MSAE-RNNs)的深度学习模型.SAE结合卷积及池化技术分别从RGB-D图像的RGB图、灰度图、深度图、3D曲面法线中提取低层次的平移不变性特征,作为多个固定树RNNs的输入,得到更加抽象的高层特征,融合后的多模态特征,采用SVM分类器进行分类.在RGB-D数据集上的实验结果表明,该算法的物体识别率达到90.7％,较其它算法提高了3％ 9％,能很好完成RGB-D物体的识别.%For the problems such as insufficient feature learning and lower accuracy of inter-class similar object recognition in cur rent RGB-D object recognition methods,a deep learning model called multimodal sparse auto-encoder and recursive neural networks (MSAE RNNs),which based on a combination of the sparse auto-encoder (SAE) and recursive neural networks (RNNs)for learning features was proposed.The SAE integrating convolution and pooling technology were used to extract low-level transnationally invariant features which were then taken as inputs to multiple and fixed-tree RNNs to compose higher order features.The multi-modal feature representations learnt from RGB images,gray images,depth images and 3D surface normal maps were sent to a SVM classifier for classification.Experimental results on RGB-D dataset demonstrate that the recognition accuracy of the proposed method for RGB-D objects can reach 90.7％oo.Moreover,compared with other methods,the proposed method improves the recognition rate by 3 ％ 9 ％,and completes the RGB-D object recognition commendably.