Information Divergences in Local Variational Approximation of Bayesian Posterior Distribution

摘要

Local variational method is a technique to approximate intractable posterior distributions in Bayesian learning. In this article, we show that the objective functions in the local variational approximation are decomposed into the sum of the Kullback information and the Bregman divergence from the approximating posterior distribution to the Bayesian posterior distribution. This provides a generic insight into this approximation method. Based on a geometrical argument in the space of approximating posteriors, we propose an efficient method to evaluate an upper bound of the marginal likelihood. We demonstrate its effectiveness through the application to the event rate estimation.