Asymptotics of Gaussian Regularized Least-Squares

摘要

We consider regularized least-squares (RLS) with a Gaussian kernel. We prove that if we let the Gaussian bandwidth sigma ->infinity while letting the regularization parameter lambda ->0, the RLS solution tends to a polynomial whose order is controlled by the relative rates of decay of 1/sigma(exp2) and lambda : if lambda = sigma (exp- (2k+1)), then, as sigma ->infinity the RLS solution tends to the kth order polynomial with minimal empirical error. We illustrate the result with an example.