A Novel Modification on the Levenberg-Marquardt Algorithm for Avoiding Overfitting in Neural Network Training

摘要

In this work, a novel modification on the standard Levenberg-Marquardt (LM) algorithm is proposed for eliminating the necessity of the validation set for avoiding overfitting, thereby shortening the training time while maintaining the test performance. The idea is that training points with smaller magnitudes of training errors are much liable to cause overfitting and that they should be excluded from the training set at each epoch. The proposed modification has been compared to the standard LM on three different problems. The results shown that even though the modified LM does not use the validation data set, it reduces the training time without compromising the test performance.