3D Data Sensing for Hand Pose Recognition

摘要

In this work we used the Kinect? sensor in order to obtain tridimensional information to perform hand pose recognition. This recognition was used to implement a system that identifies all the hand poses of the Mexican Sign Language (MSL) alphabet. We used the fusion information that provides the IR and RGB cameras in order to determinate the finger's positions and assign a skeleton to the 3D data that belongs to the hands. We take into account the distances between a reference point and the phalanges as feature to distinguish among the symbols of the MSL. In order to perform hand pose recognition with the system, a three-layer neural network with backpropagation learning was implemented. The system was tested in real time with a user different from the one used to train the system, obtaining a recognition ratio of 90.27%.