CURT MapReduce: Caching and Utilizing Results of Tasks for MapReduce on Cloud Computing

摘要

When cloud computing works for applications such as iterative applications, geographic map rendering applications, and news or URL ranking applications in order to process big data with MapReduce, it has many chances to process duplicate or similar datasets appearing in input data or intermediate data. Since processing duplicate datasets wastes many resources in clouds, cloud computing can be enhanced by the proposed CURT MapReduce system, i.e. a MapReduce system capable of caching and utilizing results of tasks, in order to avoid overheads of executing tasks to process duplicate datasets. According to real experiment observations of GREP, Radix Sort and Word Count in this paper, cloud computing gets great performance improvement from the help of the CURT MapReduce system in comparison to the native MapReduce system.