The classical multivariate linear regression problem assumes p predictor variables X1, X 2,..., Xp and a response vector y, each with n observations, and a linear relationship between the two: y = Xbeta + z, where z ∼ N(0, sigma 2). This thesis find that when p > n, there is a breakdown point for standard model selection schemes, such that model selection only works well below a certain critical complexity level depending on n/p. This notion is applied to some standard model selection algorithms (Classical Forward Stepwise, Forward Stepwise with False Discovery Rate thresholding, Lasso, LARS, and Stagewise Orthogonal Pursuit) in the case where p ≫ n.; The notion of the Phase Diagram is borrowed from signal processing and statistical physics to discover that (1) the breakdown point is well-defined for random X-models and low noise, (2) increasing noise shifts the breakdown point to lower levels of sparsity, and reduces the model recovery ability of the algorithm in a systematic way; and (3) below breakdown, the size of coefficient errors follows the theoretical error distribution for the classical linear model.; Our results are exhibited in a chemometric application using P. J. Brown. T. Fearn, and M. Vannucci's near-infrared spectroscopy data.
展开▼
