Flexible all-to-all data redistributionmethods for grid-based particle codes

摘要

The article proposes a fine-grained all-to-all communication operation that can implement flexible data redistribution patterns of irregular applications, such as particle codes. The flexibility is achieved by user-defined distribution functions, which are used to specify how data elements are to be redistributed among parallel processes on a distributed memory platform. The usage is illustrated for a particle data redistribution step of a grid-based particle code in which the destination processes for particles are calculated from the particle positions by a specific distribution function. Additionally, the fine-grained all-to-all communication operation proposed allows the duplication and modification of data elements during the data redistribution. This functionality is useful for automatically creating ghost particles for the domain decomposition of the particle code during the particle data redistribution step. The interface of the fine-grained all-to-all communication operation is described and several algorithms for implementing the operation on top of existingMPI operations are presented. Performance results on an IBM Blue Gene/Q platform demonstrate the performance of the communication operation proposed with synthetic benchmark data as well as with a parallel particle code.