Supervised Learning via Euler's Elastica Models

摘要

This paper investigates the Euler's elastica (EE) model forhigh-dimensional supervised learning problems in a functionapproximation framework. In 1744 Euler introduced the elasticaenergy for a 2D curve on modeling torsion-free thin elasticrods. Together with its degenerate form of total variation (TV),Euler's elastica has been successfully applied to low-dimensional data processing such as image denoising and imageinpainting in the last two decades. Our motivation is to applyEuler's elastica to high-dimensional supervised learningproblems. To this end, a supervised learning problem is modeledas an energy functional minimization under a new geometricregularization scheme, where the energy is composed of a squaredloss and an elastica penalty. The elastica penalty aims atregularizing the approximated function by heavily penalizinglarge gradients and high curvature values on all level curves.We take a computational PDE approach to minimize the energyfunctional. By using variational principles, the energyminimization problem is transformed into an Euler-Lagrange PDE.However, this PDE is usually high-dimensional and can not bedirectly handled by common low-dimensional solvers. Tocircumvent this difficulty, we use radial basis functions (RBF)to approximate the target function, which reduces theoptimization problem to finding the linear coefficients of thesebasis functions. Some theoretical properties of this new model,including the existence and uniqueness of solutions anduniversal consistency, are analyzed. Extensive experiments havedemonstrated the effectiveness of the proposed model for binaryclassification, multi-class classification, and regressiontasks. color="gray">