On the Activation Function and Fault Tolerance in Feedforward Neural Networks

摘要

Considering the pattern classification/recogni- tion tasks, the influence of the activation function on fault tole- ance property of feedforward neural networks is empirically in- vestigated. The simulation results show that the activation func- tion largely influences the fault tolerance and the generalization property of neural networks. It is found that, neural networks with symmetric sigmoid activation function are largely fault tol- erant than the networks with asymmetric sigmoid function. How- ever the close relation between the fault tolerance and the gener- alization property was not observed and the networks with asym- metric activation function slightly generalize better than the net- works with the symmetric activation function.