Neural network classifiers using a hardware-based approximate activation function with a hybrid stochastic multiplier

摘要

Li et al. present a novel approach for optimizing neural network implementations, that is, "a new architecture of stochastic neural networks" with a hidden approximate activation function and a hybrid stochastic multiplier that substantially reduce the hardware costs of the implementation. The paper thoroughly outlines stochastic computing and describes stochastic neural networks that provide low hardware costs and low recognition error rates. The proposed approach defines "a hardware-oriented approximate activation function," that is, the approximate sigmoid function, and explains it in a detailed way. To reduce hardware costs, a hybrid stochastic multiplier replaces the matrix multiplier; it decreases the number of inputs required for the binary parallel counters of the network. Furthermore, the authors propose "a new stochastic neuron with matrix multiplications and an approximate activation function"; they explain its activation along with its output—"a bit stream going through a comparator." The paper also discusses stochastic implementations of the multilayer perceptron, the restricted Boltzmann machine, and convolutional neural networks.