A multistage algorithm for best-subset model selection based on the Kullback-Leibler discrepancy

摘要

The selection of a best-subset regression model from a candidate family is a common problem that arises in many analyses. The Akaike information criterion (AIC) and the corrected AIC () are frequently used for this purpose. AIC and are designed to estimate the expected Kullback-Leibler discrepancy. For best-subset selection, both AIC and are negatively biased, and the use of either criterion will lead to the selection of overfitted models. To correct for this bias, we introduce an "improved" AIC variant, , which has a penalty term evaluated using Monte Carlo simulation. A multistage model selection procedure , which utilizes , is proposed for best-subset selection. Simulation studies are compiled to compare the performances of the different model selection methods.