Performance aspects of mapping neural networks onto a massively parallel SIMD computer

摘要

Abstract: In this paper we present and compare three differentmassively parallel implementations of multilayerfeedforward neural networks on a MasPar MP-1216, aparallel SIMD computer with 16,384 processors. Formultilayer feedforward networks we have obtainedsustained rates of up to 348 MCPS and 129 MCUPS withbackpropagation, a high mark for general purpose SIMDcomputers. After a brief introduction to SNNS, thepaper first focuses on the problems of mapping neuralnetworks to parallel hardware. Different aspects ofparallelism are presented. Two combinations of unit andtraining pattern parallelism were implemented as wellas link and training pattern parallelism. We describethe implementation problems in obtaining highpropagation rates on a SIMD machine and problems withthe resulting learning algorithms in general.!24