Preconditioning the Lasso for sign consistency

摘要

Sign consistency of the Lasso requires the stringent irrepresentable condition. This paper examines whether preconditioning can circumvent this condition. Let $mathbf{X}inmathbb{R}^{nimes p}$ and $Yinmathbb{R}^{n}$ satisfy the standard linear regression equation. Instead of computing the Lasso with $(mathbf{X},Y)$, preconditioning first left multiplies by $Finmathbb{R}^{nimes n}$ and then computes the Lasso with $(Fmathbf{X},FY)$. While others have proposed preconditioning for other purposes, we provide the first results that show $Fmathbf{X}$ can satisfy the irrepresentable condition even when $mathbf{X}$ fails to satisfy the condition. Preconditioning the Lasso creates a new estimator that is sign consistent in a wider variety of settings. Importantly, left multiplying the regression equation by $F$ does not change $eta$, the vector of unknown coefficients. However, left multiplying this equation by $F$ often inflates the variance of the errors. We propose a class of preconditioners to balance these costs and benefits.