An Open-Source Based Speech Recognition Android Application for Helping Handicapped Students Writing Programs

摘要

We describe in this paper how to use open-source speech recognition technologies to design and implement an Android application that helps students with physical disabilities write programs in classrooms. Google Voice Recognition (GVR)[ ], which is a free and open Android tool, is utilized to convert the speech of a user to text. To fully utilize GVR, the Android phone has to be connected to the Internet. In a typical setup, a handicapped student sits in front of a workstation with a large computer screen like the one shown in Figure 6. The student speaks to an Android phone, which converts speech to text using GVR. The text is then sent to the workstation through Wi-Fi for parsing and analysis. The processed text, which is a code segment of a program, is displayed on the workstation screen. Since the Android main activity thread cannot handle too many activities, we save the text of the speech in a buffer and use another thread, named communication thread, to send it to the workstation using the standard socket API. The producer-consumer paradigm is employed to synchronize the generation of text data by the main activity thread and the sending of the data by the communication thread[ , , ]. The server program that runs on the workstation is written in C/C++. The main thread listens at a port. When it detects data, it creates two threads to handle the data. One thread created reads in the data, parses them into words, and puts the words in a circular queue. The other thread, named processing thread, simultaneously retrieves words from the queue, processes them to generate a code segment, saves the code in a file and displays it on the screen. A condition variable[ , ] is used to synchronize the tasks between these two threads. The keywords and symbols of the programming language that the student is using, which are saved in a file are loaded into a table. The processing thread uses a hashing and mapping scheme to obtain the proper keywords and symbols from the table; as humans often speak with inconsistency, several different words may map to the same keyword. For example, when one tries to say the word import, they may say it slightly different from the standard pronunciation and the recognizer generates the word important. The scheme will map important to the same location as import to retrieve the correct keyword.