Accuracy controlled iterative method for efficient sigmoid function approximation

摘要

The sigmoid activation function plays an essential role in implementing neural networks in hardware. However, due to a high degree of non-linearity, the hardware implementation of the activation function itself is a daunting job. In this Letter, the authors proposed an accuracy controlled iterative algorithm for the piecewise linear approximation of the sigmoid function. The proposed algorithm approximates sigmoid activation with an optimised number of linear functions for a given error. The accuracy and hardware complexity of the resulting approximation function are compared with the state-of-the-art literature.