Selection of Active Predictors for Misspecified Binary Model

摘要

Selection of active predictors in high dimensional regression problems plays a pivotal role in contemporary data mining and statistical inference. However, properties of frequently applied selection procedures such as consistent choice of an active set usually strongly rely on assumption that data follows a specific model. In the presentation we address this problem and discuss general setups when estimation procedures can appproximately recover the direction of the true vector of parameters and estimate its support consistently. This explains sometimes observed phenomenon that certain procedures work well even when the underlying data generating mechanism is misspecified; e.g. methods constructed for linear models are applied to binary regression. The basic reasoning was discovered long ago by D. Brillinger and P. Rudd but it is scarcely known in data mining community. As a particular application we introduce a two-stage selection procedure which first screens predictors using LASSO method for logistic regression and then choses the final model via optimization of Generalized Information Criterion on ensuing hierarchical family. We discuss its properties and in particular the fact that in the case of misspecification it picks with large probability a model which approximates Kullback-Leibler projection (in the average sense) onto the family of logistic regressions.