A method of entropy minimization is proposed to address the author's hypothesis that multiple aspects of the learning can be described by the entropy minimization. To demonstrate the hypothesis, the competition among units such as a winner-take-all rule and selective responses is examined. In the winner-take-all rule, units compete with each other, and only one unit wins the competition, and is turned on. The process can be described by entropy minimization. In a state of minimum entropy, only one unit is turned on, while all the other units are turned off. Neurons selectively respond to specific patterns. These selective responses can also be described by using an entropy function. In a state of minimum entropy, a unit responds only to one specific pattern, while in a state of maximum entropy, the unit responds to all the patterns uniformly. Formulating two kinds of entropy function for the competition and selection, the method is applied to several problems. In all cases, it is confirmed that the entropy method enables units to compete with each other and could significantly improve the selectivity of units.
展开▼
