Comparison of Checkpointed Aided Parallel Execution Against Mapreduce

摘要

Researchers have been actively working for the past few decades in parallelizing programs so as to cut through massive data chunks for faster response. Current day processors are faster and have more number of cores. So as to utilize the computational capabilities of the processors to its full extend, processes need to be run in parallel. A task can be performed in lesser time by using parallel programming. But writing a parallel programming manually is a difficult and time consuming task. So we have to use tools to convert a sequential program to a parallel one automatically. Open-MP (Open Multi-Processing) is a set of directives which can be used to generate parallel programs written in c, c++, FORTRAN to efficient parallel programs. A new paradigm called CAPE (Check-pointing Aided Parallel Execution) is introduced that uses check-pointing technique to generate parallel programs from sequential programs provided with Open-MP directives. Map-reduce is a programming model for performing parallel processing. In this paper we have compared the performance and coding complexity of map-reduce against CAPE under different levels of difficulties.