In this paper, we extend Bayesian-Kullback Ying-Yang (BKYY) learning into a much broader Bayesian Ying-Yang (BYY) learning system via different separation functionals instead of using only Kullback divergence, and elaborate the power of BYY learning as a general learning theory for parameter learning, scale selection, structure evaluation, regularization and sampling design. Improved criteria are proposed for selecting number of densities on finite mixture and Gaussian mixtures, for selecting number of clusters in MSE clustering, for selecting subspace dimension in PCA related methods, for selecting number of expert nets in mixture of experts and its alternative model and for selecting number of basis functions in RBF nets. Three categories of non-Kullback separation functionals namely convex divergence, L/sub p/ divergence and decorrelation index, are suggested for BYY learning as alternatives for those learning models based on Kullback divergence, with some properties discussed. As examples, the EM algorithms for finite mixture, mixture of experts and its alternative model are derived with convex divergence.
展开▼
机译:在本文中,我们通过不同的分离功能而不是仅使用Kullback散度,将Bayesian-Kullback Ying-Yang(BKYY)学习扩展到更广泛的Bayesian Ying-Yang(BYY)学习系统,并详细阐述了BYY学习的强大功能用于参数学习,量表选择,结构评估,正则化和抽样设计的学习理论。提出了改进的标准,用于选择有限混合和高斯混合上的密度数,在MSE聚类中选择聚类数,在PCA相关方法中选择子空间维,在专家混合及其选择模型中选择专家网数以及在RBF网络中选择基本函数的数量。对于BYY学习,提出了三类非Kullback分离函数,即凸散度,L / sub p /散度和去相关指数,作为基于Kullback散度的学习模型的替代方法,并讨论了一些特性。例如,利用凸散度推导了有限混合,专家混合及其替代模型的EM算法。
展开▼