Machine Speech Chain with Deep Learning

摘要

In this paper, we develop a closed-loop speechchain model based on deep learning and construct asequence-to-sequence model for both ASR and TTStasks as well as a loop connection between thesetwo processes. The sequence-to-sequence model inclosed-loop architecture allows us to train our modelon the concatenation of both labeled and unlabeleddata. While ASR transcribes the unlabeled speechfeatures, TTS attempts to reconstruct the originalspeech waveform based on text from ASR. In theopposite direction, ASR also reconstructs the origi-nal text transcription given the synthesized speech.To the best of our knowledge, this is the rst deeplearning model that integrates human speech per-ception and production behaviors.