Efficient Implementation of Convolutional Neural Networks with End to End Integer-Only Dataflow

摘要

Linear INT8 quantization is presented to construct an end to end integer-only dataflow for efficient inference of modern CNNs. The INT8 method is implemented with unified layer representation, thus quantized CNNs can be partitioned into computation subgraphs consisting of stacked unified layers with simplified integer-only arithmetic flow and scaling back mechanism, indicating high effectiveness for specific hardware realization. Experimental results show that both the classification and object detection models quantized by proposed INT8 method suffer approximate 1% accuracy loss, exhibiting comparable results with TensorRT. As a result, the deep learning accelerator (DLA) with integer-only dataflow and efficient memory hierarchy is designed for CNN applications.