To stop learning using the evidence

摘要

Theoretical and practical aspects of Multi-Layer Perceptron (MLP) learning methods from the bayesian perspective were addressed for the first time by David MacKay in 1991. In this framework, the learning algorithm is an iterative process which alternates optimization of weights and estimation of hyperparameters, such as weight decay parameters. Moreover, trained MLPs that generalize better have higher "evidence", a probability which quantifies how well a MLP is adapted to a problem. This paper suggests and motivates a new methodology that computes the evidence during learning for different MLP configurations. Such estimations and the confidence intervals on test set are used to rank MLP configurations and, then, to stop learning. This learning strategy is illustrated on classification problems.