P-SFA: Probability based Sigmoid Function Approximation for Low-complexity Hardware Implementation

摘要

A probability-based sigmoid function approximation (P-SFA), which is based on piecewise linear function and neuron's values statistical probability distribution in each layer, is proposed to lower the complexity of neural network hardware implementation with only addition circuit. The sigmoid function is divided into three fixed regions, and the number of sub-regions with different sizes in each fixed region is adapted to neuron's values distribution in each layer to reduce the error between the sigmoid function and P-SFA function. The experimental results on FPGA show that the P-SFA function is efficient in terms of power and speed, and the recognition accuracies in DNN and CNN for MNIST with P-SFA are the highest among the state-of-the-art methods, up to 97.46% and 99.02% respectively. (C) 2020 Elsevier B.V. All rights reserved.