RDMA Method for MPI_REDUCE/MPI_ALLREDUCE on Large Vectors

摘要

Methods, systems and computer programs for distributing a computing operation among a plurality of processes and for gathering results of the computing operation from the plurality of processes are described. An exemplary method includes the operations of pairing a plurality of processes such that each process has a maximum of one interaction partner, selecting half of the data located at a process, dividing the selected half of the data into a plurality of data segments, transmitting a first data segment resulting from the dividing operation form the process to the interaction partner of the process, receiving a second data segment at the process from the interaction partner, concurrently with the transferring and receiving operations, performing a computing operation on a third data segment previously received from a previous interaction partner and a fourth data segment from the data segments, and iterating over the transmitting, receiving and computing operations until all the data segments have been exchanged.