Multilevel Parallelism for CFD Codes on Heterogeneous Platforms

摘要

High-level parallel programming approaches have recently become popular in complex fluid dynamics research since they are cross-platform and easy to implement. OpenACC is a high-level directive-based parallel library for offloading program execution onto a graphics processing unit (GPU). Program development for both CPU and GPU platforms can be effectively unified using the capability of portable programming with OpenACC. The directive-based model stands in contrast with the detailed implementation model of CUDA and OpenCL that suffers from poor portability and maintainability with changes in the accelerator hardware. In this work, we put effort toward drawing some outlines on efficiently annotating the base serial CPU version of the CFD code to achieve acceptable multithreaded computational performance on the GPU. An Artificial Compressibility Method (ACM) is used for studying steady-state incompressible 2D and 3D heat and fluid flows. We study loop scheduling with careful attention given to the limited on-chip memory available. The possibility of asynchronous calculations in the CFD algorithm is investigated to increase the computing performance. The PGI Fortran 15.7 compiler is used for the following study. Memory management and loop scheduling considerations result in approximately 20% speedup in the GPU computing performance from the base OpenACC implementation. A performance analysis of the thermal flow solver demonstrates that a single NVIDIA Tesla C2075 GPU card exhibits comparable speed to 8 cores on a dual-socket Xeon E5-2687W processor workstation. A multi-GPU performance analysis is performed using NVIDIA Tesla M2050 GPUs on the Virginia Tech HokieSpeed supercomputer. A weak scalability analysis demonstrates up to 92% efficiency with 32 GPUs. With 16 GPUs the thermal flow solver works up to 190 times faster than a single core of a Xeon E5645 processor, 10 times faster than a single GPU, and 12 times faster than 16 CPUs distributed on separate sockets using an MPI multi-CPU implementation of the code.