Improving the Generalization Performance of Multi-Layer-Perceptrons with Population-Based Incremental Learning

摘要

Based on Population-Based Incremental Learning (PBIL) we present a new approach for the evolution of neural network architectures and their corresponding weights. The main idea is to use a probability vector rather than bit strings to represent a population of networks in each generation. We show that crucial issues of neural network training can effectively be integrated into the PBIL framework. First, a Quasi-Newton method for local weight optimization is integrated and the moving average update rule of the PBIL is extended to continuous parameters in order to transmit the best network to the next generation, Second, and more important, we incorporate cross-validation to focus the evolution towards networks with optimal generalization performance. A comparison with standard pruning and stopped-training algorithms shows that our approach effectively finds small networks with increased generalization ability.