FlowS:一种MapReduce数据流公平调度方法

摘要

MapReduce Job scheduling has been paid great attention in academic research. Based on the analysis of Map-Reduce dataflow scheduling model, this paper presented a fair scheduling method for MapReduce dataflow-FlowS. This method can not only provide the isolation of MapReduce dataflow through dataflow pools,but also assure the fairness of resource allocation through dynamic construction algorithm. The results of experiences show that the proposed method can improve the processing efficiency of Hadoop Clusters.%MapReduce Job的调度机制一直是学术研究的热点.在分析MapReduce数据流调度模型的基础上,提出一种面向MapReduce数据流的公平调度方法FlowS.该方法采用数据流池来分配资源以保证MapReduce数据流的隔离性,并且采用数据流池动态构建算法来确保资源的公平分配.实验表明,该调度方法可以有效提高Hadoop集群对MapReduce数据流的处理效率.