Implementation of neural networks on massive memory organizations

摘要

Simulations of artificial neural networks (ANNs) on serial machines have proved to be too slow to be of practical significance. It was realized that parallel machines would have to be used to exploit the inherent parallelism in these models. The SIMD architecture presented has n PEs and n/sup 2/ memory modules arranged in an n*n array. This massive memory is used to store the weights of the neural network being simulated. It is shown how networks with sparse connectivity among neurons can be simulated in O((n+e)/sup 1/2/) time, where n is the number of neurons and e the number of interconnections in the network. Preprocessing is carried out on the connection matrix of the sparse network, resulting in data movement that has an optimal asymptotic time complexity and a small constant factor.