Nonsmooth regression and state estimation using piecewise quadratic log-concave densities

摘要

We demonstrate that many robust, sparse and nonsmooth identification and Kalman smoothing problems can be studied using a unified statistical framework. This framework is built on a broad sub-class of log-concave densities, which we call PLQ densities, that include many popular models for regression and state estimation, e.g. ℓ1, ℓ2, Vapnik and Huber penalties. Using the dual representation for PLQ penalties, we review conditions that permit interpreting them as negative logs of true probability densities. This allows construction of non-smooth multivariate distributions with specified means and variances from simple scalar building blocks. The result is a flexible statistical modelling framework for a variety of identification and learning applications, comprising models whose solutions can be computed using interior point (IP) methods. For the special case of Kalman smoothing, the complexity of this method scales linearly with the number of time-points, exactly as in the quadratic (Gaussian) case.