Diversity amplification and data generation of Chinese Sign Language based on Generative Adversarial Network

摘要

There are many factors affecting the effectiveness and accuracy of Sign Language Recognition (SLR) based on wearable device combing sEMG and IMU signals. Among them, the diversity of sign language signals caused by various factors will greatly affect the effect of SLR in the process of sign language acquisition and the use of SLR system. In the algorithm design stage, such diversity and difference should be taken into account, so that the designed algorithm can be more robust to these factors. Therefore, it is necessary to design some schemes for data amplification and data generation to solve the problem of sign language data diversity. In this paper, a random core extraction method is proposed for fast data amplification according to the characteristics of time migration without deformation. And considering that a lot of manpower and time are often consumed in the actual data acquisition process, the data volume is not sufficient. We proposed a method of sign language data generation based on Generative Adversarial Networks (GAN). This data amplification method can not only solve the problem of temporal diversity of sign language data set, but also effectively prevent model overfitting by increasing the sample numbers of sign language data set. In the experimental part, the effectiveness of the method is proved by comparing and analyzing the corresponding experimental results in the process of data generation and amplification.