A LIPREADING SYSTEM FOR JAPANESE LANGUAGE BASED ON THE HYPERCOLUMN NEURAL NETWORK

摘要

Each letter in the Japanese language ends with vowel character. This unique feature is used in the proposed lipreading system in this paper. Where, the continuous sequence of input images are quantized into discrete sequence of vowels and used to recognize different spoken sentences. This quantization process is achieved using the Hypercolumn neural network model (HCM), which consists of hierarchical layers of the Hierarchical Self-Organizing Maps (HSOM) neural network arranged as the cell planes of the Neocognitron (NC) neural network. HCM can recognize images with variant objects size, position, and spatial resolution. Results show that the system perform well in the on-line recognition of six different Japanese sentences.