When Networks Disagree: Ensemble Methods for Hybrid Neural Networks.

摘要

This paper presents a general theoretical framework for ensemble methods of constructing significantly improved regression estimates. Given a population of regression estimates, we construct a hybrid estimator which is as good or better in the MSE sense-than any estimator in the population. We argue that the ensemble method presented has several properties: It efficiently uses all the networks of a population - none of the networks need be discarded. It efficiently uses all the available data for training without over-fitting. It inherently performs regularization by smoothing in functional space which helps to avoid over-fitting. It utilizes local minima to construct improved estimates whereas other neural network algorithms are hindered by local minima. It is ideally suited for parallel computation. It leads to a very useful and natural measure of the number of distinct estimators in a population. The optimal parameters of the ensemble estimator are given in closed form. Experimental results are provided which show that the ensemble method dramatically improves neural network performance on difficult real-world optical character recognition tasks.... Generalized ensemble method, Hybrid networks, Over-Fitting, Jackknife method, Local minima.