Information-Maximization Clustering: Analytic Solution and Model Selection

摘要

A recently-proposed information-maximization clustering method (Gomes et al., NIPS2010) learns a kernel logistic regression classifier in an unsupervised manner so that mutual information between feature vectors and cluster assignments is maximized. A notable advantage of this approach is that it only involves continuous optimization of a logistic model, which is substantially easier than discrete optimization of cluster assignments. However, this method still suffers from two weaknesses: (i) manual tuning of kernel parameters is necessary, and (ii) finding a good local optimal solution is not straightforward due to the strong non-convexity of logistic-regression learning. In this paper, we first show that the kernel parameters can be systematically optimized by maximizing mutual information estimates. We then propose an alternative information-maximization clustering approach using a squared-loss variant of mutual information. This novel approach allows us to obtain clustering solutions analyti­cally in a computationally very efficient way. Through experiments, we demonstrate the usefulness of the proposed approaches.