Real-time one-shot learning gesture recognition based on lightweight 3D Inception-ResNet with separable convolutions

摘要

Gesture recognition is a popular research field in computer vision and the application of deep neural networks greatly improves its performance. However, the general deep learning method has a large number of parameters preventing the practical application on resource-limited devices. Meanwhile, collecting large number of training samples is usually time-consuming and difficult. To this end, we propose a lightweight 3D Inception-ResNet to extract discriminative features for real-time one-shot learning gesture recognition which aims to recognize gestures successfully given only one training sample for each new class. For efficient extraction of gesture features, we firstly extend the original 2D Inception-ResNet to the 3D version and then apply two kinds of separable convolutions as well as some other design strategies to reduce the number of parameters and computation complexity making it running in real-time even on CPU for feature extraction. Moreover, the consumption of storage space is also greatly reduced. In order to obtain robust performance for one-shot learning recognition, we employ an evolution mechanism by updating the root sample with innovation of new samples to enhance and improve the performance of the nearest neighbor classifier. Meanwhile, we propose an update strategy of the dynamic threshold to deal with the problem of threshold selection in real-world applications. In order to improve the robustness of recognition performance, we conduct artificial data synthesis to augment our collected dataset. A series of experiments conducted on public datasets and our collected dataset demonstrate the effectiveness of our approach to one-shot learning gesture recognition.