MORA: A Generative Approach to Extract Spatiotemporal Information Applied to Gesture Recognition

摘要

Gestures are related to a non-verbal language used on the interaction between subjects. Due to its applicability in several contexts, gesture recognition has been investigated by different researches, often investing on the capture of motion and appearance on videos. However, most of these methods do not properly explore the well-defined gesture temporal structure and are not suitable to deal with an increasing number of classes. Thus, we propose the Multi-Output Recurrent Autoencoders (MORA), an approach that relies on the representation of each gesture class independently. MORA employs a specific autoencoder model per class, composed by convolutional (3D) and a Gated Recurrent Unit (GRU) layer, what allows spatiotemporal information extraction and scalability in terms of number of classes. To validate MORA, experiments are conducted on SKIG and ChaLearn IsoGD datasets, for which the approach achieved accuracies comparable to state-of-the-art methods.