A method for connected word recognition of sign language using an image sequence

摘要

A method for connected word recognition of sign language from a color image sequence has been developed. The method is intended to be used on the sign language translation system which assists a hearing impaired person and a hearing person to communicate with each other. Since several words in conversation are expressed by a sequence of continuous action, the translation system must recognize the connected words in sign language. The sign language consists of two types of action, one is expressing a word, the other is a transient movement between words. It is important to determine the region of words out of the sequence of action in sign language. We divided the sequence into the units of movement by finding the minimum points of hand's speed. The candidates of words were determined for each segment according to the pattern of movement and the shape of a hand. After the candidates were determined, each of them was examined whether it is suitably matched with the data from the dictionary. If the coincidence was not enough, the region was considered to correspond to a transient movement. We recognized the pattern of movement of a hand by the DP matching algorithm and recognized the shape of a hand by using the BDT (Binary Decision Tree). The small number of suitable features were automatically selected by using the BDT and the words are recognized hierarchically. The recognition rate in a particular signer was 96% for 128 number of short sentences consist of two, three or four words out of 20 words.