Cross-lingual Speaker Adaptation for HMM-based Speech Synthesis based on Perceptual Characteristics and Speaker Interpolation

摘要

This paper proposes a cross-lingual speaker adaptation (CLSA) method based on perceptual characteristics (PCs). To develop a CLSA system, a state mapping (SM) based method has been recently proposed. This method extracts speaker characteristics directly from source acoustic features as linear transforms, and applies them to target models. However, it is difficult to completely eliminate language dependency from speaker transforms, and the synthesized speech is affected by source language dependency. Furthermore, there is no theoretical framework for adapting prosodic information. Hence, this paper presents a framework that uses a language-independent space of PCs for providing mapping between languages. Since the output speech is synthesized from the acoustic models of the target language only; the proposed method is-not affected by the acoustics of the source language. Moreover, by using a language-independent space for language mapping, the proposed method, is potentially able to adapt prosodic information from the target speaker. Experimental results show that the proposed method produces speech that sounds similar to the target speaker and improves the naturalness of the synthesized speech.