Evaluation of HPC-Big Data Applications Using Cloud Platforms

摘要

The path to HPC-Big Data convergence has resulted in numerous researches that demonstrate the performance trade-off between running applications on supercomputers and cloud platforms. Previous studies typically focus either on scientific HPC benchmarks or previous cloud configurations, failing to consider all the new opportunities offered by current cloud offerings. We present a comparative study of the performance of representative big data benchmarks, or "Big Data Ogres", and HPC benchmarks running on supercomputer and cloud. Our work distinguishes itself from previous studies in a way that we explore the latest generation of compute-optimized Amazon Elastic Compute Cloud instances, C4 for our experimentation on cloud. Our results reveal that Amazon C4 instances with increased compute performance and low variability in results make EC2-based cluster feasible for scientific computing and its applications in simulations, modeling and analysis.