Penalized least squares, model selection, convex hull classes and neural nets

摘要

We develop improved risk bounds for function estimation with models such as single hidden layer neural nets, using a penalized least squares criterion to select the size of the model. These results show the estimator achieves the best order of balance between approximation error and penalty relative to the sample size. Bounds are given both for the case that the target function is in the convex hull C of a class Φ of functions of dimension d (determined through empirical l_2 convering numbers) and for the case that the target is not in the convex hull.