Independent Component Analysis by General Non-Linear Hebbian-Like Rules

摘要

A number of neural learning rules have been recently proposed for IndependentComponent Analysis (ICA). In this paper, we show that in fact, ICA can be performed by very simple Hebbian on anti-Hebbian learning rules, which may have only weak relations to such information-theoretical quantities. Rather surprisingly, practically any non-linear function can be used in the learning rule, provided only that the sign of the Hebbian/anti-Hebbian term is chosen correctly. In addition to the Hebbian-like mechanism, the weight vector is constrained to have unit norm, and the data is preprocessed by prewhitening, or sphering. These results imply that one can choose the non-linearity so as to optimize desired statistical or numerical criteria.