A New Adaptive Learning algorithm to train Feed-Forward Multi-layer Neural Networks, Applied on Function Approximation Problem

摘要

Slow convergence and inverse hessian calculation respectively, are the major drawbacks of first-order and second-order learning algorithms. This paper presents a new efficient algorithm to train feed-forward Multi-Layered Perceptron (MLP) neural network that doesn't require explicit computation of the inverse Hessian matrix. Due to the use of mathematical adaptive learning rates in the purposed approach, the rating speed is improved significantly compared to the first-order algorithms. The proposed method is applied to some function approximation problems and compared with backpropagation and modified backpropagation.