Non-convex approximation based l(0)-norm multiple indefinite kernel feature selection

摘要

Multiple kernel learning (MKL) for feature selection utilizes kernels to explore complex properties of features, which has been shown to be among the most effective for feature selection. To perform feature selection, a natural way is to use the l(0)-norm to get sparse solutions. However, the optimization problem involving l(0)-norm is NP-hard. Therefore, previous MKL methods typically utilize a l(1)-norm to get sparse kernel combinations. However, the l(1)-norm, as a convex approximation of l(0)-norm, sometimes cannot attain the desired solution of the l(0)-norm regularizer problem and may lead to prediction accuracy loss. In contrast, various non-convex approximations of l(0)-norm have been proposed and perform better in many linear feature selection methods. In this paper, we propose a novel l(0)-norm based MKL method (l(0)-MKL) for feature selection with non-convex approximations constraint on kernel combination coefficients to select features automatically. Considering the better empirical performance of indefinite kernels than positive kernels, our l(0)-MKL is built on the primal form of multiple indefinite kernel learning for feature selection. The non-convex optimization problem of l(0)-MKL is further refumated as a difference of convex functions (DC) programming and solved by DC algorithm (DCA). Experiments on real-world datasets demonstrate that l(0)-MKL is superior to some related state-of-the-art methods in both feature selection and classification performance.