Stochastics of on-line back-propagation

摘要

We study on-line backpropagation and show that the existing theoretical descriptions are strictly valied only on relatively short time scales or in the vicinity of (local) minima of the backpropagation error potential. Qualitative global features (e.g., why is it much easier to escape from local minima than from global minima) may also be explained by these local descriptions, but the current approaches cannot give accurate quantitative predictions of global properties (e.g., how long does it take to reach the global minimum starting from a local minimum).