首页> 外文期刊>Operating systems review >Supporting MapReduce on Large-Scale Asymmetric Multi-Core Clusters
【24h】

Supporting MapReduce on Large-Scale Asymmetric Multi-Core Clusters

机译:在大规模非对称多核集群上支持MapReduce

获取原文
获取原文并翻译 | 示例
       

摘要

Asymmetric multi-core processors (AMPs) with general-purpose and specialized cores packaged on the same chip, are emerging as a leading paradigm for high-end computing. A large body of existing research explores the use of standalone AMPs in computationally challenging and data-intensive applications. AMPs are rapidly deployed as high-performance accelerators on clusters. In these settings, scheduling, communication and I/O are managed by general-purpose processors (GPPs), while computation is off-loaded to AMPs. Design space exploration for the configuration and software stack of hybrid clusters of AMPs and GPPs is an open problem. In this paper, we explore this design space in an implementation of the popular MapReduce programming model. Our contributions are: An exploration of various design alternatives for hybrid asymmetric clusters of AMPs and GPPs; the adoption of a streaming approach to supporting MapReduce computations on clusters with asymmetric components; and adaptive schedulers that take into account individual component capabilities in asymmetric clusters. Throughout our design, we remove I/O bottlenecks, using double-buffering and asynchronous I/O. We present an evaluation of the design choices through experiments on a real cluster with MapReduce workloads of varying degrees of computation intensity. We find that in a cluster with resource-constrained and well-provisioned AMP accelerators, a streaming approach achieves 50.5% and 73.1% better performance compared to the non-streaming approach, respectively, and scales almost linearly with increasing number of compute nodes. We also show that our dynamic scheduling mechanisms adapt effectively the parameters of the scheduling policies between applications with different computation density.
机译:具有通用和专用内核的非对称多核处理器(AMP)封装在同一芯片上,正在成为高端计算的领先范例。现有的大量研究探索了在计算挑战性和数据密集型应用中使用独立AMP的方法。 AMP作为高性能加速器快速部署在集群上。在这些设置中,调度,通信和I / O由通用处理器(GPP)进行管理,而计算则被卸载到AMP。 AMP和GPP混合集群的配置和软件堆栈的设计空间探索是一个开放的问题。在本文中,我们在流行的MapReduce编程模型的实现中探索了这个设计空间。我们的贡献是:探索AMP和GPP的混合不对称集群的各种设计替代方案;采用流方法来支持具有不对称组件的集群上的MapReduce计算;以及考虑非对称群集中各个组件功能的自适应调度程序。在整个设计过程中,我们使用双缓冲和异步I / O消除了I / O瓶颈。我们通过在具有不同计算强度程度的MapReduce工作负载的真实集群上通过实验对设计选择进行评估。我们发现,在具有资源受限和配置完善的AMP加速器的集群中,流方法与非流方法相比,性能分别提高了50.5%和73.1%,并且随着计算节点数量的增加几乎呈线性扩展。我们还表明,我们的动态调度机制可以有效地适应具有不同计算密度的应用程序之间的调度策略参数。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱:kefu@zhangqiaokeyan.com

京公网安备:11010802029741号 ICP备案号:京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有
  • 客服微信

  • 服务号