Speedup Methods for Neural Network Learning

摘要

Backpropagation is one of the most widely used learning techniques for neural networks because of its simplicity and robustness. The slowness of learning, however, is the major obstacle to its application to real-world problems. Therefore the systematic analysis of backpropagation algorithms and rapid learning methods is required. This paper presents previous research in speedup techniques of backpropagation learning, and classifies the techniques into three categories: heuristic based, numerical method based, and learning strategy based. Based on this comparative classification, some considerations needed for developing a faster learning method are discussed.