Assistance For Visually Impaired Using Finger-Tip Text Reader Using Machine Learning

摘要

Visually impaired people report large number of difficulties in their day to day life. One of the main and most important difficulty is reading texts. With the help of the latest technologies, we tend to help such difficulties by providing them with a device which could assist them in their everyday activities and also help them in studies to improve reading and learning contents. This device is a vital part in visually impaired person's life as it assists them with almost everything they come across in their typical day. This device captures image when pointed by the user and locates the text present in the image. The text is then extracted from the image and is further converted into audio to give the user with a clarified outcome. This device can be used in any paper printed texts and also handwritten texts, thus providing users an effective and efficient in time output. This project helps us identify various difficulties in detecting and recognizing text in real time by an average visually impaired person and come up with solutions to help them. In our approach we have used Fully Convolutional neural network for text level predictions and the then we use NMS to obtain the boxed geometry output of all the texts in the images. Then for the purpose of recognition the text we pass it on to tesseract OCR to obtain the extracted text, and then we convert the text to speech for the final outcome. The main motivation behind our project is to help the visually impaired people to better recognize all the text in front of them and help them live their day to day life just like any other normal person.