In this paper, we proposed a Bayesian estimator of mutual information and showed that it performs satisfactorily to solve the independence testing problem. The proposed estimator is strongly consistent even for continuous variables. Our algorithm generates a finite number of quantizations and computes the probability of the discrete sequence obtained by each quantizer. Although the number K of quantizations needs to be finite, we found the setting K = logn to be a reasonable estimation that worked well in our experiments. The proposed algorithm requires O(nlogn) computation time and O(n) memory. We compared the performance of our proposed estimator with that of the HSIC, the currently preferred independence testing principle. The idea underlying the proposed method is based on maximizing the posterior probability given the prior probability p and data x~n, y~n, although the HSIC detects abnormality assuming the null hypothesis and the given data. While we could not determine which method is superior in a general setting, we did discover that the HSIC only considers the magnitude of the data, which causes it to miss underlying relations that are not detectable by merely observing changes in magnitude. The greatest merit of our proposed algorithm compared to the HSIC is its efficiency. For the HSIC, O(n~3) computation is required for a test. Prior to testing, we need to simulate the null hypothesis and set a threshold such that the algorithm decides that the data are independent if and only if the HSIC values are less than the threshold. In this sense, executing the HSIC is time consuming. In future work, we intend to identify the exact border that is preferable with regard to the accuracy of independence testing. We also intend to publish the R program as a package.
展开▼
