Principal components identify MLP hidden layer size for optimal generalisation performance

摘要

One of the major concerns when implementing a supervised artificial neural network solution to a classification or prediction problem, is the network's performance on unseen data. The phenomenton of the network overfitting the training data, is understood and reported in the literature. Most researchers recommend a `trial and error' approach to selecting the optimal number of wights for the network, which is time consuming, or start with a large network and prune to an optimal size. Current pruning techniques basedo n approximations of the Hessian matrix of the error surface are computationally intensive and prone to severe approximation errors if a suitable minimal training error has not been achieved.