The Infomin Principle: A Unifying Information-based Criterion for Forming Topographic Mappings

摘要

In this paper, we propose a new principle "InfoMin" and a new criterion based on it for forming topographic mappings. The InfoMin principle asserts that a topographic mapping is formed by minimizing the average of information transferred through small areas of the mapping. This criterion can explain clearer why more highly correlated neurons are placed nearer. In addition, we characterize the criterion as a special case of the unifying objective function (the C measure) proposed by Goodhill and Sejnowski [3], and compare it with some topographic mapping methods based on the dimension reduction. We show that our criterion is defined just by the values of neurons and usable without any knowledge on the structure in the input space. Numerical experiments on natural scenes show that the optimization of the criterion could simulate the dimension reduction methods.