EFFICIENT AND ACCURATE WEIGHT QUANTIZATION FOR NEURAL NETWORKS

摘要

Various embodiments relate to a method for producing a plurality of weights for a neural network, wherein the neural network includes a plurality of layers, including: receiving a definition of the neural network including the number of layers and the size of the layers; and training the neural network using a training data set including: segmenting N weights of the plurality of weights into I weight sub-vectors {right arrow over (w)}(i) of dimension K=N/I; applying constraints that force sub-vectors {right arrow over (w)}(i) to concentrate near a (K−1)-dimensional single-valued hypersurface surrounding the origin; and quantizing sub-vectors {right arrow over (w)}(i) to a set of discrete K-dimensional quantization vectors {right arrow over (q)}(i) distributed in a regular pattern near the hypersurface, wherein each sub-vector {right arrow over (w)}(i) is mapped to its nearest quantization vector {right arrow over (q)}(i).