SLO-Driven Task Scheduling in MapReduce Environments

摘要

MapReduce is emerging as an important programming model for massive data processing. A key challenge in MapReduce environments is the ability to efficiently control resource allocation and task scheduling for achieving Service Level Objectives (SLOs) of MapReduce jobs. However, there are few effective task scheduling methods to guarantee MapReduce jobs' SLOs. Therefore, we address this challenge by proposing a SLO-driven task scheduling mechanism in this paper. Based on the MapReduce performance model we build, our mechanism dynamically adjusts resource allocation and task scheduling in order to guarantee the SLOs of jobs and improve global job utility. Experimental results show that our SLO-driven task scheduler effectively meets the specified job latency SLOs and enhances job utility on tested MapReduce programs.