首页>
外国专利>
SYSTEMS AND METHODS FOR EXCHANGE OF DATA IN DISTRIBUTED TRAINING OF MACHINE LEARNING ALGORITHMS
SYSTEMS AND METHODS FOR EXCHANGE OF DATA IN DISTRIBUTED TRAINING OF MACHINE LEARNING ALGORITHMS
展开▼
机译:机器学习算法的分布式训练中数据交换的系统和方法
展开▼
页面导航
摘要
著录项
相似文献
摘要
Systems and methods may make exchanging data in a neural network (NN) during training more efficient. Exchanging weights among a number of processors training a NN across iterations may include sorting generated weights, compressing the sorted weights, and transmitting the compressed sorted weights. On each Kth iteration a sort order of the sorted weights may be created and transmitted. Exchanging weights among processors training a NN may include executing a forward pass to produce a set of loss values for processors, transmitting loss values to other processors, and at each of the processors, performing backpropagation on at least one layer of the NN using loss values received from other processors.
展开▼