Constraint-aware Workflow for Automation of Big Data Analytics based on GraphPlan

摘要

The use of the big data analytics (BDA) platform is increasingly becoming prevalent in the data sciences. However, BDA processes consume resources and time excessively. Automating BDA processes is a cognitive approach to the BDA domain, which is most impaired by its heavy consumption of time and resources. However, the BDA workflow is highly dependent on diversified constraints because of the high variability, veracity, and volume of data processing to accomplish highly influential and sophisticated requirements. The workflow has to pass rigorous and diverse data mining steps; each step contains several tasks and these tasks made by many sub tasks, that it must be accomplished to progress to the next step in the workflow. This increases the available solution space for BDA processes. The intelligent heuristic approach is needed to address the domain-specific concerns which are large solution space, and awareness of constraints are caused the BDA planning. Therefore, we propose to use GraphPlan-based dynamic workflow generation for the BDA domain. Our empirical studies prove that the proposed sophisticated method satisfied planning requirements and outperformed a related planning technique.