METHOD FOR FLEXIBLE, FAST ALL-REDUCE ON ARBITRARY TREE TOPOLOGY

摘要

A method for sending data across processors to combine the data on the processors is described. In one embodiment, a method includes receiving a set of data at a set of processors configured in an asymmetric or symmetric tree topology including a root and one or more leaves. Target portions of the set of data are assigned to processors of the set of processors based on a number of child processors that are connected to a parent node. The method includes sending iteratively apportioned combined data between child processors sharing the same parent node in each branch of the tree topology starting from the one or more leaves and increasing levels in the tree topology until reaching the root. The method also includes sending the combined data between child processors from one branch to child processors in at least one other branch.