Detecting and Bypassing Trivial Computations in Convolutional Neural Networks

摘要

Convolutional neural networks (CNNs) recently are able to exceed human accuracy in various application domains such as image recognition, medical diagnosis, and financial analysis. However, the high computational complexity of CNNs incurs high energy consumption on current hardware implementations. Existing solutions such as pruning and quantization typically require retraining or fine-tuning to regain accuracy, which can be cost-prohibitive and time-consuming. This paper proposes a retraining-free approach to reducing the computation workload of CNNs during inference by detecting and bypassing the trivial computations. We define trivial computations as the computations the results of which can be determined without actual computations. The examples include multiplication with 0, +1/-1 and addition with 0 or addition of opposite numbers. Correspondingly, we develop bypass circuits to detect the trivial computations. Once detected, the circuit delivers the predetermined result without an actual computation. Experimental results on MNIST and EMNIST show that the CNNs with bypass circuits can lead to 30.66-33.52% energy savings without any accuracy loss. This technique can be used together with existing techniques such as pruning and quantization because it is totally complimentary to such techniques.