Conversation Engine for the Hearing and Vocally Impaired Using CNN

摘要

On our planet, Around nine billion individuals are hard of hearing and unable to speak. Correspondence between hard of hearing, quiet, and a typical individual has consistently been a difficult errand. Since ordinary individuals are not trained on gesture based communication, at the time of crisis passing on their message is troublesome. The human hand has always remained a popular choice to convey information in situations where other forms like speech cannot be used. Voice Conversion System (VCS) with Hand Gesture Recognition (HGR) and translation will be extremely valuable to have an appropriate discussion between an ordinary individual and a disabled individual in any language. This prescribed model intends to build up a framework that changes over the gesture-based communication into a human hearing voice in the ideal language to pass on a message to typical individuals, as well as converting speech into justifiable gesture-based communication for the tragically challenged. The Convolutional Neural Network (CNN) is used to train the model on different hand gestures and an app is built using Flask which uses this model. This application enables vocally impaired people to pass on their message using signs which gets changed over to human-understandable language and the speech is given as the result, as well as it translates the speech into hand gestures for the hearing impaired people.