A NONPARAMETRIC APPROACH TO MODEL SELECTION

摘要

We consider the problem of learning a regression function from samples based on a sequence of candidate models from which an optimal one is to be selected. In the absence of any reliable a priori information about the data generating process, we adopt a nonparametric approach to functionally characterize the data statistics. This nonparametric description is then used to derive a nonparametric reference model whose complexity is automatically determined by data-driven procedure. The nonparametric complexity can be used as a benchmark to select a suitable parametric complexity from the class of candidates. The proposed method is highly effective against overfitting and largely outperforms previous approaches in experimental study of polynomial curve fitting. The only requirement is to have access to a collection of unlabeled data.