METHOD AND APPARATUS FOR PARALLELIZING LAYERS OF DEEP NEURAL NETWORKS ONTO PARALLEL COMPUTING SYSTEMS

摘要

A computer-implemented method of creating a modified Deep Neural Network-DNN-comprises: carrying out a parallelization process to allocate computational and memory requirements of a selected layer or set of layers of the DNN across more than one computer process. The parallelization process comprises: selecting the number of computer processes to be used to execute the selected layer(s); determining if the selected number of computer processes is the same as the number of selected layers and, in dependence upon the result of the determination, assigning the selected layer(s) to the computer processes in accordance with a predetermined protocol; and modifying the DNN by: creating and adding to the DNN a first additional layer, to distribute input data from the selected layer(s) to the computer processes, and a second additional layer, to collate and combine output data from the computer processes; and setting connections between the first and second additional layers and the selected layer(s) in the DNN so as to arrange for parallel execution of the selected layer(s) by the computer processes.