Structure Learning in Conditional probability Models via an Entropic Prior and Parameter Extinction

摘要

We introduce an entropic prior for multinomial parameter estimation problems and solve for its maximum a posteriori (MAP) estimator. The prior is a bias for maximally structured and minimally ambiguous mod- els. In conditional probability models with hidden state, iterative MAP estimation drives weakly supported parameters toward extinction, effec- tively turning them off. Thus, structure discovery is folded into parame- ter estimation.