Extended Bayesian Learning

摘要

In Bayesian learning one represents the relative degree of believe in different values of the weight vector - including biases - by considering a probability distribution function over weight space. In general, this a priori probability is expected to come from a Gaussian with zero mean and flexible variance which is callded a hyperparameter. It can be optimized automatically during training by maximizing the evidence. The extended Bayesian learning (EBL) approach consists of considering a more general form of priors by using several weight classes and by considering the mean of the Gaussian distribution to be another hyperparameter. We propose an algorithm which determines automatically the optimal number of different weight classes and where the weights can change from one class to another. Our approach is applied in several benchmark problems and outperforms simple Bayesian learning as well as other optimization strategies.