Model selection in neural networks: Some difficulties

摘要

There exist several questions with regard to the selection of a best neural network for a given set of data. One such difficulty is the identification problem wherein there is an inability to replicate weight values for a trained network, and the other is the use of statistical principles in choosing the number of hidden layers and the number of hidden nodes. This paper discusses these two difficulties and presents the conclusions by experimenting on three real data sets. In Section 2, the basic model of a feedforward logistic neural network is presented. In Section 3, some of the strategies employed in model selection are discussed. In Section 4, the Taylor series approach to investigate the neural network is presented. In Section 5, experiments are conducted to compare the networks trained with polytope and back propagation algorithm, networks with single and double hidden layers and networks with logistic activation and hyperbolic tangent activation. The data, implementation issues, results and further implications are presented. The author summarizes two important results and states why the model selection strategies of neural networks still remain inconclusive. (28 refs.)