Multiagent-consensus-MapReduce-based attribute reduction using co-evolutionary quantum PSO for big data app

摘要

AbstractThe attribute reduction for big data applications has become an urgent challenge in pattern recognition, machine learning and data mining. In this paper, we introduce the multi-agent consensus MapReduce optimization model and co-evolutionary quantum PSO with self-adaptive memeplexes for designing the attribute reduction method, and propose a multiagent-consensus-MapReduce-based attribute reduction algorithm (MCMAR). Firstly, the co-evolutionary quantum PSO with self-adaptive memeplexes is designed for grouping particles into different memeplexes, which aims to explore the search space and locate the global best region during the attribute reduction of big datasets. Secondly, the four layers neighborhood radius framework with compensatory scheme is constructed to partition big attribute sets by exploiting the interdependency among multiple-relevant-attribute sets. Thirdly, a novel multi-agent consensus MapReduce optimization model is adopted to perform the multiple-relevance-attribute reduction, in which five kinds of agents are used to conduct the ensemble co-evolutionary optimization. So the uniform reduction framework of different agents’ co-evolutionary game under the bounded rationality is further refined. Fourthly, the approximation MapReduce parallelism mechanism is permitted to formalize to the multi-agent co-evolutionary consensus structure, interaction and adaptation, which enhances different agents to share their solutions. Finally, extensive experimental studies substantiate the effectiveness and accuracy of MCMAR on some well-known benchmark datasets. Moreover, successful applications in big medical datasets are expected to dramatically scaling up MCMAR for complex infant brain MRI in terms of efficiency and feasibility.