Hadoop Preemptive Deadline Constraint Scheduler

摘要

MapReduce is a programming model developed for processing large amount of data with parallel and distributed algorithm on a cluster of computing nodes. It provides convenient programming interface distributing data intensive works in a cluster environment such as Hadoop. Preemption is an effective approach for MapReduce scheduler in avoiding the delay of high priority jobs while allowing the system to be shared by regular jobs. In this paper the problem of deadline constraint scheduling on a MapReduce model is addressed. We present a new preemption approach which considers the remaining execution time of the job being executed in making the decision of preemption. Computer simulation demonstrates that the proposed scheme reduces the job execution time and waiting time in the queue compared to the existing scheme.