Second order back-propagation learning algorithm and its application for neural network

摘要

In this paper, a new second order recursive learning algorithm to multilayer feedforward network is proposed. This algorithm makes not only each layer's errors but also second order derivative information factors backpropagate. And it is proved that it is equivalent to Newton iterative algorithm and has second order convergent speed. New algorithm achieves the recurrence calculation of Newton search directions and the inverse of Hessian matrices. Its calculation complexity corresponds to that of common recursive least squares algorithm. It is stated clearly that this new algorithm is superior to Karayiannis' second order algorithm (1994) according to analysis of their properties.