Feature Selection Using Smooth Gradient L_(1/2) Regularization

摘要

In terms of L_(1/2) regularization, a novel feature selection method for a neural framework model has been developed in this paper. Due to the non-convex, non-smooth and non-Lipschitz characteristics of L_(1/2) regularizer it is difficult to directly employ the gradient descent method in training multilayer perceptron neural networks. A smoothing technique has been considered to approximate the original L_(1/2) regular-izer. The proposed method is a two-stage updating approach. First, a multilayer network model with smoothing L_(1/2)regularizer is trained to eliminate the unimportant features. Second, the compact model without regularization has been simulated until there is no improvements for the performance. The experiments demonstrate that the presented algorithm significantly reduces the redundant features while keeps a considerable model accuracy.