Towards Near Data Processing of Convolutional Neural Networks

摘要

The gap between the processing speed of the CPU and the access speed of the memory is becoming a bottleneck for many data intensive applications. This gap can be reduced if the computation can be taken near to the data. Recent advancement in memory technology has made it feasible to have 3D stacked memory along with the capability of having an integrated logic layer, thus making near data processing (NDP) feasible. Convolutional Neural Networks (CNNs) are used in a wide range of applications such as image processing, video analysis, natural language processing, etc. They are data intensive and have highly parallel computations. If we can perform them near the memory, we can achieve higher throughput. This paper proposes a CNN Logic Unit (CLU) as a hardware implementation in the logic layer associated with 3D stacked memory, e.g. the Hybrid Memory Cube (HMC) implementing the concept of NDP. Full system simulation results show that the method is very promising as it gives several times improvement for CNN operations when performed near memory as compared to in conventional CPU based systems with DRAM. We get an improvement of 76x in performance while energy consumption is reduced by 55x.