A Novel Multilevel Queue based Performance Analysis of Hadoop Job Schedulers

摘要

Objectives: In this paper, we discuss on the importance of multilevel queues in scheduling Hadoopmapreduce jobs. Methods/Statistical Analysis: Modifications are done on HDFS and yarn configuration files to suit the multilevel queues. This work constitutes the performance analysis of various existing job schedulers such as FIFO, Fair and Capacity schedulers. Findings: Significant achievements are achieved which includes performance evaluation metrics for comparative understanding of the proposed and existing techniques. The final outcome of the work demonstrates the need for multilevel queue scheduling with allocation policies and the optimal placement of jobs in queues. Application/Improvements:With the adoption of multilevel queue scheduling, there is a significant improvement in placing jobs in multilevel queues for the jobs submitted by the users.