Effect of activation function symmetry on training of SFFANNs with the backpropagation algorithm

摘要

On 17 learning task (12 function approximation and 5 real life regression problems), we compare the efficiency and efficacy of using asymmetric or anti-symmetric activation functions in sigmoidal feedforward artificial neural network training and usage. The result obtained in the experiment allows us to conclude that for networks trained using the batch update variant of the backpropagation algorithm, the usage of antisymmetric activation functions may give better performance is some cases, but in a few cases networks using antisymmetric activation functions may give better performance and in majority of cases the performance of networks using anti-symmetric or asymmetric activation are equivalent. Thus a clear preference for anti-symmetric activation functions cannot be made.