Prediction, Model selection and Random Dimension Penalties

摘要

Let Z~1,...,Z~n be i.i.d. vectors, each consisting of a response and a few explanatory variables. Suppose we have K collections of predictors, i.e., collections of functions of the explanatory variables, that predict the response variable. Given the "empirically best" predictor within each of the collections, we suggest a criterion to select a predictor from those K candidates based on minimax regret; we also show how to find an asymptotically optimal selection procedure under this criterion. We then show how the conventional setting of model selection is related to the above. Conventionally, the term 'model' refers to a collection of distributions, while its analog in our setting is a collection of predictors. The assumptions about the possible distributions of Z~i (the model) are non-parametric, while the collections of the predictors are assumed to be 'conveniently' parametrized.