Support region of μ-law logarithmic quantizers for Laplacian source applied in neural networks

摘要

The main aim of the paper is to provide effective and accurate solutions for the calculation of the support region of the mu-law logarithmic companding quantizers. A new solution for the starting point of iterative methods will be proposed, that provides very accurate value of the support region (being the main parameter needed for the design of the quantizer) only after one iteration of the iterative method. Based on this new starting point, an accurate closed-form approximate expression for the calculation of the support region will be derived, as one of the main contributions of the paper. To significantly simplify implementation of the mu-law companding quantizer, piecewise linearization is performed. A new linearization method is presented, based on the optimization of the last segments. Derivation of an accurate closed-form formula for the support region of the linearized quantizer is done, as an important contribution. The obtained linearized mu-law companding quantizer is very simple to design (due to closed-form formulas) and to implement (due to linearization), providing at the same time very high performance (due to optimization of the last segments). Due to these and other advantages (robustness, adjustability to the statistical distribution of the input signal), the proposed quantizer can be used in many topical applications, such as in receivers of 5G wireless systems or in neural networks for quantization of weights and activations. The paper provides an application of the designed quantizers for quantization of weights of a neural network, showing significant decreasing of the bit-rate compared to the standard full-precision representation (from 32 bits to just 5 bits), with the same prediction accuracy of the network.