Simple 'Linearized' Learning Algorithm Which Outperforms Back-Propagation

摘要

A class of algorithms is presented for training multilayer perceptrons usingpurely linear techniques. The methods are based upon linearizations of the network using error surface analysis, followed by a contemporary least squares estimation procedure. Specific algorithms are presented to estimate weights node-wise, layer-wise, and for estimating the entire set of network weights simultaneously. In several experimental studies, the node-wise method is superior to back-propagation and an alternative linearization method due to Azimi-Sadjadi et al. in terms of number of convergences and convergence rate. The layer and network-wise updating offer further improvement.