£_1-norm low-rank linear approximation for accelerating deep neural networks

摘要

In this paper, we develop a L-1-norm based low-rank matrix approximation method to decompose large high-complexity convolution layers into a set of low-complexity convolution layers with low-ranks to accelerate deep neural networks. Based on the alternating direction method (ADM), we derive a mathematical solution for this new L-1-norm based low-rank decomposition problem. Our experimental results on public datasets, including CIFAR-10 and ImageNet, demonstrate that this new decomposition scheme outperforms the recently developed L-2-norm based nonlinear decomposition method, which achieved the state-of-the-art performance. (C) 2020 Elsevier B.V. All rights reserved.