False Oracles and Strict MML Estimators

摘要

Minimum Message Length (MML) and Minimum Description Length (MDL) inductive inference techniques are based on the information-theoretic notion of transmitting data concisely. Both methods are universally applicable, consistent and efficient. MML is a Bayesian technique and is invariant under parameter transformations; however, it is a quadratic approximation to a slightly more efficient two-part coding technique, Strict MML (SMML), which maps from the data space onto a countable, discretized subset of the parameter space. It is shown that the posterior distribution of the parameter vector is a "false oracle" in that no fair comparison between the true ("oracular") parameter vector and a sampling from the posterior will enable us to distinguish one from the other. It is further shown that the invariant, consistent and efficient Bayesian SMML point estimation technique closelyapproximates (and converges to) a false oracle. Hence, SMML inductions are practically indistinguishable from the truth in the absence of data other than that used in the induction.