Dynamic-Stride-Net: Deep Convolutional Neural Network with Dynamic Stride

摘要

It is crucial to reduce the cost of deep convolutional neural networks while preserving their accuracy. Existingmethods adaptively prune DNNs in a layer-wise or channel-wise manner based on the input image. In thispaper, we develop a novel dynamic network, namely Dynamic-Stride-Net, to improve residual network withlayer-wise adaptive strides in the convolution operations. Dynamic-Stride-Net leverages a gating network toadaptively select the strides of convolutional blocks based on the outputs of the previous layer. To optimizethe selection of strides, the gating network is trained by reinforcement learning. The oating point operationsper second (FLOPS) is signi cantly reduced by adapting the strides to convolutional layers without loss ofaccuracy. Dynamic-Stride-Net reduces the computational cost by 35%-50% with equivalent accuracy of theoriginal model on CIFAR-10 and CIFAR-100 datasets. It outperforms the state-of-the-art dynamic networks andstatic compression methods.