Penalized Partial Least Squares with applications to B-spline transformations and functional data

摘要

We propose a novel framework that combines penalization techniques with Partial Least Squares (PLS). We focus on two important applications. (1) We combine PLS with a roughness penalty to estimate high-dimensional regression problems with functional predictors and scalar response. (2) Starting with an additive model, we expand each variable in terms of a generous number of B-spline basis functions. To prevent overfitting, we estimate the model by applying a penalized version of PLS. We gain additional model flexibility by incorporating a sparsity penalty. Both applications can be formulated in terms of a unified algorithm called Penalized Partial Least Squares, which can be computed virtually as fast as PLS using the kernel trick. Furthermore, we prove a close connection of penalized PLS to preconditioned linear systems. In experiments, we show the benefits of our method to noisy functional data and to sparse nonlinear regression models.