Penalized Regression Methods with Application to Domain Selection and Outlier Detection.

摘要

Variable selection is one of the most important problems in statistical literature, and a popular method is the penalized regression. In this dissertation, we investigate two types of variable selection problems via penalized regression. The first problem is related to the domain selection for the varying coefficient model, which identifies important regions of the varying coefficient that are related to the response. The second problem is related to variable selection problem for the linear model with the existence of outliers, and deal with variable selection and outlier detection simultaneously.;In Chapter 1, we give some introduction and background by overview of the variable selection methods, the local polynomial regression and the robust regression. In Chapter 2, we consider the varying coefficient model which allows the relationship between the predictors and response to vary across the domain of interest, such as time. In applications, it is possible that certain predictors only affect the response in particular regions and not everywhere. This corresponds to identifying the domain where the varying coefficient is nonzero. Towards this goal, we incorporate local polynomial smoothing and penalized regression into one framework. We establish asymptotic properties of our penalized estimators and show that they enjoy the oracle properties in the sense that they have the same bias and asymptotic variance as the local polynomial estimators as if the sparsity is known as a priori. The choice of appropriate bandwidth and computational algorithms are discussed. In Chapter 2, we study the outlier detection and variable selection problem in linear regression. A mean shift parameter is added to the linear model to reflect the effect of the outlier, where an outlier has a nonzero shift parameter. We then apply an adaptive regularization on these shift parameters and shrink most of them to zero. For those observations with nonzero mean shift parameter estimates, they are regarded as outliers. Meanwhile, an L1 penalty is added to the regression parameters to select important predictors. We propose an efficient algorithm to solve this jointly penalized optimization problem and use the extended BIC tuning method to select the regularization parameters since the number of parameters exceeds the sample size. Theoretical results are provided in terms of high breakdown point, full efficiency as well as outlier detection consistency. We illustrate our method with simulation and real data. Our method is extended to high-dimensional problems with p &