Multi-Threaded SIMD Implementation of the Back-Propagation Algorithm on Multi-Core Intel Xeon Processors

摘要

A combinations of using efficient algorithms and well designed implementations leads to great high performance applications. This paper show how to make the back-propagation algorithm run faster on multi-core processors and scale to the future hardware that may have more cores and faster memory. On two dual core Intel Xeon processors, each supports Hyper-Threading technology, the performance of the multi-threaded SIMD implementation of the matrix back-propagation (MBP) algorithm gives around 20 times higher than the best conventional implementation on the same hardware. On reasonably large networks, experimental results show that the use of Intel streaming SIMD extensions, matrix blocking, loop unrolling, and multithreading on eight logical processors speed up the MBP by factors of 1.4, 1.75, 1.8, 4.6, respectively. Moreover, five single-precision floating-point operations can be performed in a single clock cycle by exploiting the memory hierarchy, by executing multiple instructions from multiple threads on multiple data (MIMD), and by selecting an efficient algorithm, which is based on matrix operations (MBP algorithm) instead of matrix-vector operations (BP algorithm).