Five discrete symbiotic organisms search algorithms for simultaneous optimization of feature subset and neighborhood size of KNN classification models

摘要

This paper develops five new discrete Symbiotic Organisms Search (SOS) algorithms for simultaneous optimization of feature subset and neighborhood size of k-nearest neighbor model to improve classification accuracy. The first algorithm is a discrete version of the original SOS algorithm, named DSOS. The second is a hybrid derived from enhancing the first DSOS with paired swap local search, named DHSOS. The third is a cooperative hybrid between DSOS and a discrete particle swarm optimization (DPSO), named DSOSPSO. The fourth and fifth are modified from the second by adapting population size rather than fixing it, named APDHSOS and AP2DHSOS, respectively. Five existing metaheuristic algorithms are also implemented and extended for comparison. The performance of these algorithms employing the k-nearest neighbor classification model are evaluated in terms of classification error and computational time based on stratified k-fold cross validation with 11 datasets. The classification errors of five larger data sets are also obtained to further verify the test results. Based on the test results, it is found that: (1) feature selection with fixed neighborhood size yields lower errors than optimized neighborhood size without feature selection; (2) simultaneous optimization of feature subset and neighborhood size overall outperforms optimizing either feature subset or neighborhood size alone; and (3) among all the ten algorithms tested for simultaneous optimization, APDHSOS and AP2DHSOS emerge to tie the best in terms of classification error, implying that adaptive population size works better than fixing population size. (C) 2017 Elsevier B.V. All rights reserved.