A Comparative Study of Genetic Algorithms, Particle Swarm Optimization, and Differential Evolution in Problem of Feature Selection through Structure Retention

摘要

Recently, many feature selection methods have used population-based optimization algorithms, such as Genetic Algorithms (GA), Particle Swarm Optimization (PSO), and Differential Evolution (DE), as vehicles to address the combinatorial optimization problem of feature selection. Each of these algorithms comes with some clearly delineated advantages and limitations. Since these three algorithms are supposed to find a solution to a given objective function but use different processes and strategies; therefore, it is appropriate to compare their performance. However, very few studies have been done to compare their performance especially in terms of their computational effort, computational time and convergence rate. In this paper, we compare the performance of GA, PSO and DE optimization techniques, in order to find the best feature subset that can preserve the structure of the original data set. The experiment results show that all three algorithms give the same result when dealing with a small dimensionality of the data set. Meanwhile, if we increase the dimensionality of the data set the results show that DE is superior to the results produced by the two other algorithms. Apart from superior performance, DE is more robust, easy to implement, and quick to converge into the optimal solution.