首页>
外国专利>
HIGHLY PERFORMANT PIPELINE PARALLEL DEEP NEURAL NETWORK TRAINING
HIGHLY PERFORMANT PIPELINE PARALLEL DEEP NEURAL NETWORK TRAINING
展开▼
机译:高度性能的管道平行深度神经网络培训
展开▼
页面导航
摘要
著录项
相似文献
摘要
Layers of a deep neural network (DNN) are partitioned into stages using a profile of the DNN. Each of the stages includes one or more of the layers of the DNN. The partitioning of the layers of the DNN into stages is optimized in various ways including optimizing the partitioning to minimize training time, to minimize data communication between worker computing devices used to train the DNN, or to ensure that the worker computing devices perform an approximately equal amount of the processing for training the DNN. The stages are assigned to the worker computing devices. The worker computing devices process batches of training data using a scheduling policy that causes the workers to alternate between forward processing of the batches of the DNN training data and backward processing of the batches of the DNN training data. The stages can be configured for model parallel processing or data parallel processing.
展开▼