An Empirical Study on Regression Algorithms

摘要

Regression is one of the most important tasks in the machine learning and data mining. In recent years, a number of learning algorithms for regression have been introduced. Unfortunately, there are few projects that compared the performance of these methods by experiment In this paper, we present a empirical comparison between ten algorithms used for regression: k-nearest neighbor (KNN), it-nearest-neighbor with distance weighted (KNNDW), linear regression (LR), locally weighted linear regression (LWLR), model trees (M5P), reduced error pruning tree (REPTree), naive Bayes (NB), genetic programming (GP), back propagation (BP), and support vector machine (SVM). We test these ten different regression algorithms on the whole 36 data sets recommended by Weka. The ten-folds cross validation (CV) criteria is used to compare these algorithms. Finally we test the experimental results via two-tailed t-test with significantly different probability of 95% to compare the performance of each pair of algorithms in terms of the relative absolute error.