Repeated comprehensibility maximization in competitive learning

摘要

In this study, we propose a new type of information-theoretic method in which the comprehensibility of networks is progressively improved upon within a course of learning. The comprehensibility of networks is defined by using mutual information between competitive units and input patterns. When comprehensibility is maximized, the most simplified network configurations are expected to emerge. Comprehensibility is defined for competitive units, and the comprehensibility of the input units is measured by examining the comprehensibility of competitive units, with special attention being paid to the input units. The parameters to control the values of comprehensibility are then explicitly determined so as to maximize the comprehensibility of both the competitive units and the input units. For the sake of easy reproducibility, we applied the method to two problems from the well-known machine learning database, namely, the Senate problem and the cancer problem. In both experiments, any type of comprehensibility can be improved upon, and we observed that fidelity measures such as quantization errors could also be improved.