Class Confusability Reduction in Audio-Visual Speech Recognition Using Random Forests

摘要

This paper presents an audio-visual speech classification system based on Random Forests classifiers, aiming to reduce the intra-class misclassification problems, which is a very usual situation, specially in speech recognition tasks. A novel training procedure is proposed, introducing the concept of Complementary Random Forests (CRF) classifiers. Experimental results over three audio-visual databases, show that a good performance is achieved with the proposed system for the different types of input information considered, viz., audio-only information, video-only information and fused audio-video information. In addition, these results also indicate that the proposed method performs satisfactorily over the three databases using the same configuration parameters.