Optimal Unsupervised Learning in Feedforward Neural Networks

摘要

We investigate the properties of feedforward neural networks trained with Hebbian learning algorithms. A new unsupervised algorithm is proposed which produces statistically uncorrelated outputs. The algorithm causes the weights of the network to converge to the eigenvectors of the input correlation with largest eigenvalues. If a network trained in this way is used as input to a layer trained using the Widrow-Hoff (LMS) algorithm, the system implements an optimal Wiener filter. The algorithm is closely related to the technique of Self-supervised Backpropagation, as well as other algorithms for unsupervised learning. The algorithm to texture processing, image coding, and stereo depth edge detection are given. It is shown that the algorithm can lead to the development of filters qualitatively similar to those found in primate visual cortex. Keywords: Neural nets; Artificial intelligence; Pattern recognition. (SDW)