Quality improvement of HMM-based synthesized speech based on decomposition of naturalness and intelligibility using asymmetric bilinear model with non-negative matrix factorization

摘要

HMM-based synthesized voices are intelligible but not natural especially in limited data condition due to over-smoothing of speech spectra. One solution for this problem is to use voice conversion techniques to convert over-smoothed spectra into natural spectra. Although conventional conversion techniques can transform speech spectra to natural ones in order to improve naturalness, they cause unexpected distortions on acceptable intelligibility of synthesized speech. The aim of this paper is to improve naturalness without violating acceptable intelligibility employing a novel asymmetric bilinear model using non-negative matrix factorization (NMF) to separate the naturalness and intelligibility of the synthesized speech. Subjective evaluations carried out on English dataset confirm that the achieved synthesis quality is higher than other methods in limited data condition and competitive in large data condition. Moreover, non-negativity constrain in NMF helps to explain the physical meanings of factored matrices as follows: the first matrix describes naturalness and the second describes intelligibility of the speech.