Bias approximations for likelihood-based estimators

摘要

Bias approximation has played an important role in statistical inference, and numerous bias calculation techniques have been proposed under different contexts. We provide a unified approach to approximating the bias of the maximum likelihood estimator and the l(2) penalized likelihood estimator for both linear and nonlinear models, where the design variables are allowed to be random and the sample size can be a stopping time. The proposed method is based on the Woodroofe-Stein identity and is justified by very weak approximations. The accuracy of the derived bias formulas is assessed by simulation for several examples. The bias of the ridge estimator in high-dimensional settings is also discussed.