An effect of initial distribution covariance for annealing Gaussian restricted Boltzmann machines

摘要

In this paper, we investigate an effect that the covariance of an initial distribution for annealed importance sampling (AIS) exerts on the estimation accuracy for the partition functions of Gaussian restricted Boltzmann machines (RBMs). A common choice for an AIS initial distribution is a Gaussian RBM (GRBM) with zero weight connections. Such an initial distribution does not show any covariance between variables. However, target distributions generally allow a finite covariance between variables. We propose a method to design the covariance matrix of an initial distribution for GRBMs. We empirically analyze the effect of the initial distribution covariance on the estimation accuracy of AIS. The proposed method for designing initial distributions outperforms conventional methods under various conditions.