Analysis of Stopping Criteria in Artificial Neural Networks

摘要

The goal of this study was to decide when to terminate training of an artificialneural network (ANN). In pursuit of this goal, several characteristics of the ANN were monitored throughout ANN training: classification error rate (of the training set, testing set, or a weighted average of the two): moving average classification error rate: measurements of the difference between ANN output and desired output (error sum of squares, total absolute error. or largest absolute error); or ANN weight changes (absolute weight change, squared weight change, or relative weight change). Throughout this research, the learning rate was held constant at 0.35. The momentum was not used because the primary interest of this study was evaluating when to terminate training as opposed to the speed of reaching a decision. The ANN structure was held constant with two input features, two output classes, and one hidden layer. Finally, the practice of training after processing each exemplar was followed. Results indicated three conclusions. First, multiple runs are required for ANN analysis because ANNs are not guaranteed to converge to the same solution. Second, the classification error rate, a moving average of the classification error rate, and the total absolute error (all computed on the testing set) provided a similar final classification. Third, once the stopping criteria functions cease to decrease, select a set of ANN final weights at random. On average, they yielded as low as or lower classification error rate and variance of the classification error rate than other possible choices. Artificial neural networks, stopping criteria.