Sign language recognition based on adaptive HMMS with data augmentation

摘要

Vision based sign language recognition (SLR) is a challenging task due to the complexity of signs and limited data collection. To improve the recognition precision, this paper proposes an adaptive GMM-based (Gaussian mixture model) HMMs (Hidden Markov Models) framework. We discover that inherent latent states in HMMs are not only related to the number of key gestures and body poses, but also related to the kinds of their translation relationships. We propose adaptive HMMs and obtain the hidden state number for each sign with affinity propagation clustering. Furthermore, to enrich the training dataset, we propose a data augmentation strategy by adding Gaussian random disturbances. Experiments on a vocabulary of 370 signs demonstrate the effectiveness of our proposed method over the comparison algorithms.