H2T: A Simple Hadoop-to-Twister Translator for Cloud Computing

摘要

MapReduce has become one of the most popular programming model for big data analysis in cloud systems due to its simplicity for implementing data parallel applications. There are several platforms for users to develop their applications based on MapReduce framework such as Hadoop and Twister. Hadoop is one of the most popular runtime systems for MapReduce applications and supported by various organizations, however, the original design for Hadoop did not propose an iterative feature efficiently which is required for many scientific applications. Twister, another system for Iterative MapReduce, is introduced and designed to facilitate iterative applications based on MapReduce framework. It has shown that Twister has the better performance than Hadoop on some applications such as Pair wise Distance Calculation (Smith Waterman Gotoh distance). Automatic translations between two program languages in cloud platforms can help developers move their applications from one cloud to anther cloud without changing codes. In this paper, we propose a simple Hadoop-to-Twister translator named H2T which is designed for converting simple Hadoop applications into Twister applications. The experimental results show that translated Twister applications is much faster than original Hadoop applications.