'DATA TEMPERATURE' IN MINIMUM FREE ENERGIES FOR PARAMETER LEARNING OF BAYESIAN NETWORKS

摘要

Maximum likelihood method for estimating parameters of Bayesian networks (BNs) is efficient and accurate for large samples. However, the method suffers from over-fitting when the sample size is small. Bayesian methods, which are effective to avoid over-fitting, present difficulties for determining optimal hyper-parameters of prior distributions with good balance between theoretical and practical points of view when no prior knowledge is available. As described in this paper, we propose an alternative estimation method of the parameters on BNs. The method uses a principle, rooted in thermodynamics, of minimizing free energy (MFE). We define internal energies, entropies, and temperature, which constitute free energies. Especially for temperature, we propose a "data temperature" assumption and some explicit models. This approach can treat the maximum likelihood principle and the maximum entropy principle in a unified manner of the MFE principle. For assessments of classification accuracy, our method shows higher accuracy than that obtained using the Bayesian method with normally recommended hyper-parameters. Moreover, our method exhibits robustness for the choice of introduced hyper-parameters.