Stochastic Complexities for Mixture of Exponential Family in Variational Bayes Approach

摘要

The Variational Bayes learning, proposed as an approximation of the Bayesian learning, has provided computational tractability and good generalization performance in many applications. However, little has been done to investigate its theoretical properties. In this paper, we discuss the Variational Bayes learning of the mixture of exponential families and derive the upper and lower bounds of the stochastic complexities or the marginal likelihoods. We show that the stochastic complexities become smaller than those of regular statistical models, which means the advantage of the Bayesian learning still remains in the Variational Bayes learning.