MLE’s Bias Pathology, Model Updated MLE, and Wallace’s Minimum Message Length Method

摘要

The inherent bias pathology of the maximum likelihood estimation method is confirmed for models with unknown parameters θ and ψ when maximum likelihood estimate (MLE) ψ̂ is function of MLE θ̂. To reduce ψ̂'s bias the likelihood equation to be solved for ψ is updated using the model for the data Y in it. For various models with ψ a shape parameter model updated (MU) MLE, ψ̂, reduces ψ̂'s bias. For the Pareto model ψ̂ reduces in addition ψ̂'s variance. The results explain the difference that puzzled Fisher, between biased ψ̂ and the unbiased estimate he obtained for two models with the abandoned two-stage procedure used in MUMLE's implementation. ψ̂ is also obtained with the minimum message length method thus motivating the use of priors in frequentist inference.