A Weighted Multi-task Learning Approach for Mandarin-English Code-switching Speech Recognition

摘要

This work proposes a weighted Multi-task Learning (wMTL-DNN) approach for enhancing the performance of Mandarin-English Code-Switching Speech Recognition (MECS-SR). Focusing on Mandarin, a special language, this approach chooses two acoustic units to obtain different kinds of information for acoustic modeling. One is initial/final-based units, and the other is phoneme-based. Those two acoustic units can describe the salient acoustic and phonetic information for Mandarin. To make full use of the important information, we adopt Multi-task Learning framework based on Deep Neural Network (MTL-DNN), which can fuse those information simultaneously. To further get a just weight during transferring knowledge, we propose a weighted MTL-DNN (wMTL-DNN) to fuse these different information. Extensive experiments are carried out on LDC2015S04 and Mixed Error Rate (MER) is used as performance metric for the Code-switching Speech Recognition. Compared with the baseline and the first MECS-SR system [1] on LDC2015S04, MER of proposed approach is relatively reduced by 11.69% and 12.43%, respectively. Even relatively reduced by 11.23%, compared with the first MECS- SR system [1] added language identification task.