首页> 外文期刊>Concurrency, practice and experience >On-demand data co-allocation with user-level cache for grids
【24h】

On-demand data co-allocation with user-level cache for grids

机译:按需数据与用户级缓存的网格共分配

获取原文
获取原文并翻译 | 示例

摘要

Conventional remote data access middlewares usually provide client applications with either a pre-staging scheme or an on-demand access scheme to fetch data. The pre-staging scheme uses parallel downloads to fetch a completed input file from multiple data sources, even when only a tiny file fragment is required. Such a transfer scheme consumes unnecessary data transmission time and storage space. In contrast, the on-demand scheme downloads only the required data blocks from a single data source and does not fully utilize the downstream bandwidth of the computing nodes. This paper presents a middleware called 'Spigot' that facilitates legacy (grid-unaware) applications to transparently access remote data by using native I/O function calls. Spigot uses the on-demand concept to avoid unnecessary data transfer and adopts a co-allocation download algorithm to improve the data transfer performance. Moreover, it uses the pre-fetching strategy to reduce the data waiting time by overlapping data acquisition and data processing. It also provides the client application with its own user-level cache, which is advantageous since a larger cache space is available in comparison with the kernel-level cache. Further, it is easy to maintain data consistency between Spigot nodes. The experimental results indicate that Spigot achieves superior performance in reducing the data waiting time than the pre-staging and the on-demand access schemes.
机译:常规的远程数据访问中间件通常为客户端应用程序提供预升级方案或按需访问方案以获取数据。预分阶段方案使用并行下载从多个数据源中获取完整的输入文件,即使仅需要很小的文件片段也是如此。这样的传输方案消耗了不必要的数据传输时间和存储空间。相反,按需方案仅从单个数据源下载所需的数据块,而没有完全利用计算节点的下游带宽。本文介绍了一种名为“ Spigot”的中间件,该中间件可帮助传统(无网格)应用程序通过使用本机I / O函数调用透明地访问远程数据。 Spigot使用按需概念来避免不必要的数据传输,并采用了协同分配下载算法来提高数据传输性能。此外,它使用预取策略通过重叠数据采集和数据处理来减少数据等待时间。它还为客户端应用程序提供了自己的用户级缓存,这是有利的,因为与内核级缓存相比,它具有更大的缓存空间。此外,很容易维持Spigot节点之间的数据一致性。实验结果表明,Spigot在减少数据等待时间方面比预分级和按需访问方案具有更高的性能。

著录项

  • 来源
    《Concurrency, practice and experience》 |2010年第18期|p.2488-2513|共26页
  • 作者单位

    Institute of Computer and Communication Engineering, Department of Electrical Engineering, National Cheng Kung University, No. 1, Ta-Hsueh Rd., Tainan City 701, Taiwan;

    rnDepartment of Computer Science and Applications, Leader University, No. 188, Sec. 5, Au-Chung Rd., Tainan City 709, Taiwan;

    rnInstitute of Computer and Communication Engineering, Department of Electrical Engineering, National Cheng Kung University, No. 1, Ta-Hsueh Rd., Tainan City 701, Taiwan;

    rnInstitute of Computer and Communication Engineering, Department of Electrical Engineering, National Cheng Kung University, No. 1, Ta-Hsueh Rd., Tainan City 701, Taiwan;

  • 收录信息
  • 原文格式 PDF
  • 正文语种 eng
  • 中图分类
  • 关键词

    data Grid; on-demand access; data co-allocation download; pre-fetching; user-level cache; distributed shared memory (DSM);

    机译:数据网格;按需访问;数据协同分配下载;预取用户级缓存;分布式共享内存(DSM);

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱:kefu@zhangqiaokeyan.com

京公网安备:11010802029741号 ICP备案号:京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有
  • 客服微信

  • 服务号