Acoustic modeling with a shared phoneme set for multilingual speech recognition without code-switching

摘要

This paper proposes a new acoustic modeling method for the automatic speech recognition (ASR) of data, in which multilingual utterances are mixed, without using any language identification technologies. To perform ASR of unknown-language utterance, first, language identification is performed to determine the language. Then, a language-specific ASR system is used to recognize the utterance. Our proposed method does not train language-specific acoustic models but trains an acoustic model that can speech-recognize utterances spoken by some sort of language. To realize multilingual acoustic modeling, we create a new phoneme set by sharing a part of language-specific phonemes with other languages. The shared phoneme set enables the amount of training data to increase on appearance. Therefore, the acoustic model with the shared phoneme set can perform ASR for a minor language (low-resource language) utterance. The experimental result showed that the acoustic model with the shared phoneme set improved ASR performance for a few languages in comparison with the language-specific ASR system in which language identification was perfectly performed.