双通道Faster R-CNN在RGB-D手部检测中的应用

摘要

In most vision tasks related to human hands,such as human computer interaction and sign language recognition,hand detection is a distinctly important preprocessing phase.With the development of RGB-D data acquisition equipment,the extra depth data can complement the color data effectively,so they can provide more powerful feature representation.The traditional detection methods based on hand-crafted features(skin color or HOG) cannot form a well hand representation.While a lot of detection methods based on deep learning can avoid such weakness by learning effective features from data.To combine the advantages of RGB-D data and deep learning,a two-stream Faster R-CNN detection framework was proposed in this paper.The proposed method adds an extra depth stream information,and combines it with RGB stream information in the feature level.The experiment results show that the proposed method can achieve a higher detection precision than the Faster R-CNN framework which uses RGB or fuses the RGB and Depth in the data level.Thus,the proposed method can fuse the color and depth data effectively,and improve the performance of hand detection.%在人机交互、手语识别等大量与人手有关的视觉任务中,手部检测是极为重要的一个预处理阶段.随着RGB-D数据采集设备的发展,额外提供的深度数据能够与传统使用的彩色数据互相补充以提供更强的特征表达.此外,传统的检测方法由于使用肤色、HOG等手工设计的特征,不能对手部进行很好的表达.而基于深度学习的检测方法通过从数据中自动学习有效的特征避免了这个问题.为了结合RGB-D数据和深度学习技术的优点,提出了一种融合彩色和深度数据的双通道Faster R-CNN检测框架.该方法在原有Faster R-CNN检测框架的基础上,增加了Depth通道信息,并在特征层面上将其与RGB通道信息进行融合.实验结果表明,所提方法在性能上比仅采用RGB或在数据层面上融合的Faster R-CNN框架有明显优势.因此,该方法能有效融合来自彩色和深度通道的数据,以提升手部检测性能.