基于PAC-Bayes边界理论的SVM模型选择方法

摘要

PAC-Bayes risk bound integrating theories of Bayesian paradigm and structure risk minimization for stochastic classifiers has been considered as a framework for effective evaluating the generalization capability of machine learning algorithms. Aiming at the problem of model selection of SVM, this paper analyzes the theoretical framework of PAC-Bayes bound and its application to SVM, and combines the PAC-Bayes bound with grid search method based on cross validation. A method of model selection based on PAC-Bayes bound(PBB-GS)is put forward to select the best penalty parameter and kernel parameter rapidly. From the experimental results of the UCI datasets, it draws the conclusion that the parameters selected by PBB-GS can make SVM achieve better generalization performance, and this method is simple, fast and accu-rate, which can improve the model selection of SVM effectively.%PAC-Bayes边界理论融合了贝叶斯定理和随机分类器的结构风险最小化原理，它作为一个理论框架，能有效评价机器学习算法的泛化性能。针对支持向量机（SVM）模型选择问题，通过分析PAC-Bayes边界理论框架及其在SVM上的应用，将PAC-Bayes边界理论与基于交叉验证的网格搜索法相结合，提出一种基于PAC-Bayes边界的SVM模型选择方法（PBB-GS），实现快速优选SVM的惩罚系数和核函数参数。UCI数据集的实验结果表明该方法优选出的参数能使SVM具有较高的泛化性能，并具有简便快速、参数选择准确的优点，能有效改善SVM模型选择问题。