Mapping of neural network models onto massively parallelhierarchical computer systems

摘要

Investigates the proposed implementation of neural networks onmassively parallel hierarchical computer systems with hypernet topology.The proposed mapping scheme takes advantage of the inherent structure ofhypernets to process multiple copies of the neural network in thedifferent subnets, each executing a portion of the training set.Finally, the weight changes in all the subnets are accumulated to adjustthe synaptic weights in all the copies. An expression is derived toestimate the time for all-to-all broadcasting, the principal mode ofcommunication in implementing neural networks on parallel computers.This is later used to estimate the time required to execute variousexecution phases in the neural network algorithm, and thus to estimatethe speedup performance of the hypernet in implementing neural networks