Learning Useful Representations Through Stacked Self-Organizing Maps

摘要

In this work we explore an original strategy for building deep networks, based on stacking layers of Self-Organizing Maps (SOM) with finite weights. We aim to show that our approach, with enough hidden variables, is capable to represents any "soft" distribution over the visible variables, where "soft" means that the distribution does not contain any probabilities of 1 or 0. The algorithm compute the model one layer at a time. The first layer receives the input observations and compute a probability of membership for each observation and each neuron. These probabilities become the input of the second layer, which compute a new set of probabilities, and so on. The number of neurons decrease in each layer after the first. The proposed algorithm is experimentally tested on artificial and real data-sets. The effect of the added hidden layers for the representation of data structure is experimentally demonstrated.