Variational inference as iterative projection in a Bayesian Hilbert space with application to robotic state estimation

摘要

Variational Bayesian inference is an important machine learning tool that finds application from statistics to robotics.The goal is to find an approximate probability density function (PDF) from a chosen family that is in some sense“closest” to the full Bayesian posterior. Closeness is typically defined through the selection of an appropriate lossfunctional such as the Kullback-Leibler (KL) divergence. In this paper, we explore a new formulation of variationalinference by exploiting the fact that (most) PDFs are members of a Bayesian Hilbert space under careful definitionsof vector addition, scalar multiplication, and an inner product.We show that, under the right conditions, variationalinference based on KL divergence can amount to iterative projection, in the Euclidean sense, of the Bayesian posterioronto a subspace corresponding to the selected approximation family. We work through the details of thisgeneral framework for the specific case of the Gaussian approximation family and show the equivalence to anotherGaussian variational inference approach.We furthermore discuss the implications for systems that exhibit sparsity,which is handled naturally in Bayesian space, and give an example of a high-dimensional robotic state estimationproblem that can be handled as a result. We provide some preliminary examples of how the approach could beapplied to non-Gaussian inference and discuss the limitations of the approach in detail to encourage follow-on workalong these lines.