神威太湖之光上OpenFOAM的移植与优化

摘要

The Sunway TaihuLight supercomputer based on the Chinese-designed many-core processors is the world's fastest system with a peak performance of 125.4 PFlops.OpenFOAM (open source field operation and manipulation) is one of the most popular open source computational fluid dynamics (CFD) software which is written in C++ and not fully compatible with compilers on the heterogeneous many-core processor SW26010.This paper ported OpenFOAM based on SW26010's MPE(management processing element)/CPE (computing processing element) cluster architecture.To overcome the compilation incompatibility problem,we adopted the mixed-language application design.We also applied several SW26010's feature-specific optimizations on the hotspot of OpenFOAM to deliver high performance,such as the register communication,vectorization,and double buffering.The experiments on SW26010 using real datasets show that the single-CG (core group) code runs 8.03x faster than the well-tuned version on the MPE,and the performance of single-CG is 1.18x higher than the serial implementation of Intel(R) Xeon(R) CPU E5-2695 v3.We also optimized the parallel implementation of OpenFOAM and yielded speedups of 184.9x on 256 CGs.The porting methods and optimizations presented can also be referenced for other complex C-+-+ programs to achieve high performance on SW26010.%神威太湖之光是最新一期Top500榜单上排名第一的超级计算机,峰值性能为125.4 PFlops,其计算能力主要归功于国产SW26010众核处理器.OpenFOAM (Open Source Field Operation and Manipulation)是计算流体力学领域使用最广泛的开源软件包,但是由于其基于C++实现,与神威太湖之光上的异构众核处理器SW26010的编译器不兼容,因此无法直接在该架构上有效运行.基于SW26010的主核/从核的体系架构移植了OpenFOAM的核心计算代码,并采用混合语言编程实现的方式来解决编译不兼容的问题.此外,通过寄存器通信、向量化和双缓冲等优化手段,单核组的性能较优化后的主核代码提高了8.03倍,较Intel(R) Xeon(R) CPU E5-2695 v3的串行执行性能提高了1.18倍.同时,将单核组的实现扩展到了神威太湖之光的大规模集群上,并进行了强可扩展性测试,256个核组上实现了184.9倍的加速.采用的移植方式和优化手段也可以为其他复杂C++程序在神威太湖之光上的应用提供借鉴.