Parallelization of Back Propagation Neural Network on Knights Landing Platform

摘要

Artificial Neural Network is an effective technique of prediction needing a lot of time for training. Benefitting from data parallelization and structure parallelization, training time of network has been greatly reduced. In this paper, a hybrid parallel algorithm is proposed to further optimize training process. The experimental results indicate that the parallel algorithm owns ideal expansibility and great speed-up ratio up to 52.9 on KNL. When optimized with SIMD instruction set and MCDRAM, the maximum speedup ratio comes to 124.21.