LEVERAGING DIRECTED ACYCLIC GRAPH (DAG) INFORMATION TO GROUP TASKS FOR EXECUTION

摘要

Embodiments for leveraging directed acyclic graph (DAG) information to group tasks for execution, by at least one processor device. For a set of tasks, an input host set is determined for each task using a shuffle type and hosts used during a previous stage, and the tasks in the set of tasks determined to be within a same input host set are classified together. An Input/Output (I/O) cost for each task in the set of tasks is calculated and tasks within the set of tasks are grouped into task groups according to an applied allocation time. The task groups are launched commensurate with a calculated delay time from each input host set so as to overlap I/O phases and central processing unit (CPU) phases to improve CPU efficiency, the launched task groups being executed by an executor according to the calculated delay time.