Reducing the Inter-speaker Variance of CNN Acoustic Models Using Unsupervised Adversarial Multi-task Training

摘要

Although the Deep Neural Network (DNN) technology has brought significant improvements in automatic speech recognition, the technology is still vulnerable to changing environmental conditions. The adversarial multi-task training method was recently proposed to increase the domain and noise robustness of DNN acoustic models. Here, we apply this method to reduce the inter-speaker variance of a convolutional neural network-based speech recognition system. One drawback of the baseline method is that it requires speaker labels for the training dataset. Hence, we propose two modifications which allow the application of the method in the unsupervised scenarios; that is, when speaker annotation is not available. Our approach applies unsupervised speaker clustering, which is based on a standard feature set in the first case, while in the second case we modify the network structure to perform speaker discrimination in the manner of a Siamese DNN. In the supervised scenario we report a relative error rate reduction of 4%. The two unsupervised approaches achieve smaller, but consistent improvements of about 3% on average.