Sign language recognition through kinect based depth images and neural network

摘要

Sign language is the language of the people with hearing and speaking disabilities. In it mostly hands are moved in a particular way which along with some facial expression produces a meaningful thought which the speaker would like to convey to others. Using the sign language people with speaking and hearing disabilities can communicate with others who know the language very easily but it becomes difficult when it comes to interacting with a normal person. As a result there is a requirement of an intermediate system which will help in improving the interaction between people with the hearing disabilities as well as with the normal people. In this paper we present a sign language recognition technique using kinect depth camera and neural network. Using the kinect camera we obtain the image of the person standing in front of the camera and then we crop the hand region from the depth image and pre-process that image using the morphological operations to remove unwanted region from the hand image and find the contour of the hand sign and from the particular contour position of the hand we generate a signal on which Discrete Cosine Transform (DCT) is applied and first 200 DCT coefficient of the signal are feed to the neural network for training and classification and finally the network classify and recognize the sign. A data set of sign from 0 to 9 are formed using kinect camera and we tested on 1236 images in the database on which training is applied and we achieved 98% training and an average accuracy for all the sign recognition as 83.5%.