A framework for big data pre-processing and search optimization using HMGA-ACO: a hierarchical optimization approach

摘要

The huge potential associated with big data has prompted a developing research field that has rapidly attracted huge enthusiasm from differing sectors. Unlike traditional databases, advanced for quick access and summarization of structured data and all around characterized inquiries, big data is accepted to fill in as a raw material for the creation of new knowledge. We look at the complexity placed by big search spaces, dominated by the number of variables and domain of each variable, in search and optimization problems. While an extensive, even unbounded, search area disables the effectiveness and efficiency of search, a complex structure of constraints additionally increases the difficulty in that the search space becomes highly unpredictable. In order to overcome the above issues we propose a novel Hierarchical Manipulated Genetic Algorithm with Ant Colony Optimization with data pre-processing which can possibly improve the data and recover the data with more accuracy and precision. The proposed hierarchical optimization can help to boost the speed of search, and the exertion of search is reduced with the utilization of pre-processing.