Analysis of Hierarchical Variational Bayes Approach in Linear Inverse Problem

摘要

It is known that, in singular models, the Bayes estimation commonly has the advantage of generalization performance over the maximum likelihood estimation, however, its accurate approximation requires huge computational costs. The variational Bayes (VB) approach has been proposed as a tractable approximation method of the Bayes estimation, and shown good generalization performance in many applications. Recently, the VB approach has been applied to the automatic relevance determination model (ARD), a kind of hierarchical Bayesian learning, in brain current estimation from MEG data, a practical linear inverse problem. On the other hand, we have been proved that, in three-layer linear neural networks (LNNs), the VB approach is asymptotically equivalent to the James-Stein type shrinkage estimation, and theoretically clarified its generalization performance. In this paper, noting the similarity between the ARD in a linear problem and an LNN, we analyze a simplified version of the VB approach in the ARD. We will show its relation to the shrinkage estimation, and the fact that the relevance determination is caused by a kind of phase transition.