Parameter Optimization of Polynomial Kernel SVM from miniCV

摘要

Polynomial kernel support vector machine (SVM) is one of the most computational efficient kernel-based SVM. Implementing an iterative optimization method, sequential minimal optimization (SMO) makes it more hardware independent. However, the test accuracy is sensitive to the values of hyperparameters. Moreover, polynomial kernel SVM has four hyperparameters which complicate cross-validation in parameter optimization. In this research, we transform polynomial kernels to have bounded values and analyze the relations between hyperparameters and the test error rate. Based on our discoveries, we propose mini core validation (miniCV) to fast screen out an optimized hyperparameter combination especially for large datasets. The proposed miniCV is a parameter optimization approach completely built on the distribution of the data generated via the iterative SMO training process. Since miniCV depends on the kernel matrix directly, it saves miniCV from cross-validation to optimize hyperparameters in kernel-based SVM.