Neural network based feature extraction for speech and image recognition

摘要

This work investigates features derived from an artificial neural network. These artificial neural network based probabilistic features have become a major component of current state-of-the-art systems for automatic speech recognition and other areas, e.g. image recognition. A detailed study of the artificial neural network based features helps to improve the feature extraction and to understand which information of the speech signal is relevant for recognition. Two algorithms are investigated which are widely used to integrate the information derived from an artificial neural network: the tandem and the hybrid approach. This work studies the effect of each of the algorithms in terms of recognition performance w.r.t. word error rate and the computational requirements. In addition, a detailed comparison and a discussion of the main advantages of each integration approach are given. Furthermore, novel extensions are proposed improving the artificial neural network feature extraction and the final recognition performance of the systems trained. These extensions concern the input features and the topology of the network used to train the artificial neural network and are independent of the integration method. Different short-term and long-term features model other complementary aspects of the speech signal. By combining these different feature sets the development circle of the speech recognition system can be simplified. This allows increasing the model complexity of the artificial neural network or of the acoustic model. The topology of an artificial neural network has a huge impact on the quality of the features derived from the artificial neural network. This work investigates the hierarchical framework, the bottle-neck processing and recurrent neural networks, especially the long-short-term-memory structure and the training of bi-directional networks. Furthermore, this work examines cross-lingual artificial neural network features and their impact on the topology and the amount of audio data used to train such features. The training and testing language of the artificial neural network features differs and the system development circle is simplified when such cross-lingual artificial neural network based features are used. In addition, this work analyses different supervised and unsupervised weight pre-training techniques. The initialization of the weights of a deep neural network is critical since the optimization function is non-convex. A new unsupervised pre-training technique is developed which allows the optimization of the loss function directly and provides a clear stopping criterion compared to other pre-training techniques like Restricted Boltzmann Machines. Finally, this work analyzes the generality of the artificial neural network based feature extraction approach by transferring the concept to different image tasks, optical character recognition and automatic sign language recognition. While most results are confirmed, some surprising new results are obtained.