Domain-Based MapReduce Programming Model for Complex Scientific Applications

摘要

The MapReduce programming model has introduced simple interfaces to a large class of applications. Its easy-to-use APIs and autonomic parallelization are attracting attentions from scientific community. However, current MapReduce-style scientific frameworks focus more on the most popular MapReduce applications that can be easily partitioned and involve little communication across map or reduce tasks. They typically lack adequate support for more complex applications that involve iterative communication and dynamic domain partitioning. In this paper, via abstraction of numerical domains of many applications, we present a domain-based MapReduce programming model for iterative and dynamic scientific applications. Using real-world applications, we introduce a general methodology to adapt APIs of legacy scientific codes into the more developer-friendly MapReduce-like programming model.