Improving Neural Network Performance for Character and Fingerprint Classificationby Altering Network Dynamics

摘要

The paper shows that performance equal to or better than the Probabilistic NeuralNetwork (PNN) can be achieved with a single three-layer Multilayer Perceptron (MLP) by making fundamental changes in the network optimization strategy. These changes are: (1) Neuron activation functions are used which reduce the probability of singular Jacobians; (2) Successive regularization is used to constrain volume of the weight space being minimized; (3) Boltzmann pruning is used to constrain the dimension of the weight space; and (4) Prior class probabilities are used to normalize all error calculations so that statistically significant samples of rare but important classes can be included without distortion of the error surface. All four of these changes are made in the inner loop of a conjugate gradient optimization iteration and are intended to simplify the training dynamics of the optimization. On handprinted digits and fingerprint classification problems these modifications improve error-reject performance by factors between 2 and 4 and reduce network size by 40% to 60%.