Adaptive on-line optimizing the Gaussian kernel for classification based on the kernel target alignment

摘要

Kernel target alignment is a very efficient evaluation criterion. It has been widely applied in kernel optimization. However the traditional kernel methods that based on the Kernel target alignment optimize the kernel function mainly with batch gradient descent algorithm. This kind of methods has to scan through the entire training set at each step, which is much too costly. The On-line learning algorithm exactly solve above problem. At each step, on-line learning algorithm only need one example then discarded after learning, which make on-line learning algorithm fast, simple, and often make few statistical assumptions. Thus, in this paper, we propose a novel method to optimize the Gaussian kernel with on-line learning. We formulate the objective criterion for kernel optimization based on kernel target alignment. The objective criterion can be proved to have a determined global minimum point. Then, we use the on-line learning algorithm to optimize the formulated kernel function. In addition, in order to get an appropriate learning rate for the algorithm to accelerate the convergence rate, we use an adaptive rate learning method to optimize the kernel function. Finally, we evaluate the empirical performance of the proposed kernel optimization method on ten diverse datasets. The experimental results show that the proposed method is more effective than the state-of-the-art kernel optimization algorithms.