Variable selection and estimation using a continuous approximation to the L-0 penalty

摘要

Variable selection problems are typically addressed under the regularization framework. In this paper, an exponential type penalty which very closely resembles the L-0 penalty is proposed, we called it EXP penalty. The EXP penalized least squares procedure is shown to consistently select the correct model and is asymptotically normal, provided the number of variables grows slower than the number of observations. EXP is efficiently implemented using a coordinate descent algorithm. Furthermore, we propose a modified BIC tuning parameter selection method for EXP and show that it consistently identifies the correct model, while allowing the number of variables to diverge. Simulation results and data example show that the EXP procedure performs very well in a variety of settings.