Shift: A Zero FLOP, Zero Parameter Alternative to Spatial Convolutions

摘要

Neural networks rely on convolutions to aggregate spatial information.However, spatial convolutions are expensive in terms of model size andcomputation, both of which grow quadratically with respect to kernel size. Inthis paper, we present a parameter-free, FLOP-free "shift" operation as analternative to spatial convolutions. We fuse shifts and point-wise convolutionsto construct end-to-end trainable shift-based modules, with a hyperparametercharacterizing the tradeoff between accuracy and efficiency. To demonstrate theoperation's efficacy, we replace ResNet's 3x3 convolutions with shift-basedmodules for improved CIFAR10 and CIFAR100 accuracy using 60% fewer parameters;we additionally demonstrate the operation's resilience to parameter reductionon ImageNet, outperforming ResNet family members. We finally show the shiftoperation's applicability across domains, achieving strong performance withfewer parameters on classification, face verification and style transfer.