Mapping of neural network models onto massively parallel hierarchical computer systems

摘要

Investigates the proposed implementation of neural networks on massively parallel hierarchical computer systems with hypernet topology. The proposed mapping scheme takes advantage of the inherent structure of hypernets to process multiple copies of the neural network in the different subnets, each executing a portion of the training set. Finally, the weight changes in all the subnets are accumulated to adjust the synaptic weights in all the copies. An expression is derived to estimate the time for all-to-all broadcasting, the principal mode of communication in implementing neural networks on parallel computers. This is later used to estimate the time required to execute various execution phases in the neural network algorithm, and thus to estimate the speedup performance of the hypernet in implementing neural networks.