On Benchmarking the Matrix Multiplication Algorithm using OpenMP, MPI and CUDA Programming Languages

摘要

Parallel programming languages represent a common theme in the evolution of high performance computing (HPC) systems. There are several parallel programming languages that are directly associated with different HPC systems. In this paper, we compare the performance of three commonly used parallel programming languages, namely: OpenMP, MPI and CUDA. Our performance evaluation of these languages is based on the implementation of matrix multiplication algorithms. Matrix multiplication is chosen because of its wide application in many scientific and engineering problems such as bioinformatics, linear algebra, and computer graphics. Our results show that CUDA programming delivers up to 15 fold speed acceleration relative to OpenMP and MPI Programming. However, CUDA programming may prove comparatively more challenging to programmers.