One of the holy grails of unsupervised learning has been a general learning principle that could be applied in multiple stages to discover increasingly complex structures in the input. Such a learning principle could form the basis for understanding hierarchical processing in the cortes. and explain how and why complex recentpive fields such as those found in the inferotemporal cortex develop. This article presents a general statistical solution to this problem that is applicable in data-rich environments, and has connections to well-known principles of cortical self-organization. The model takes unlabelled, randomly ordered data vectors as input. Using this data, it constructs a pyramidal hierarchy of layers and successively builds increasingly complex approximations to the dataspace. Each layer in the hierarchy takes its input from the previous layer (with the first receiving the data)< learns a representation of its input, and re-encodes the input f or processing by the next layer. Three processes operate at each layer: (1) a robust vector quantization process, called the Batch Neural Gas (BaNG) algorithm, distributes cluster centers to minimize mean-squared distances to the input vectors, 92) a topological graph construction process links up the cluster centers using lateral connections into a graph that approximates the topology of the input space, and (3) an encoding method recodes the inputs with resepect to the topology of the input space using path lengths along the graph and the rank-ordered distances from cluster centers to input vectors. It is shown that such an algorithm can solve previously unsolvable, highly nonlinear problems in statistics such as separating intertwined spirals embedded in noise.
展开▼
