An Extended Algorithm Using Adaptation of Momentum and Learning Rate for Spiking Neurons Emitting Multiple Spikes

摘要

This paper presents two methods of using the dynamic momentum and learning rate adaption, to improve learning performance in spiking neural networks where neurons are modelled as spiking multiple times. The optimum value for the momentum factor is obtained from the mean square error with respect to the gradient of synaptic weights in the proposed algorithm. The delta-bar-delta rule is employed as the learning rate adaptation method. The XOR and Wisconsin breast cancer (WBC) classification tasks are used to validate the proposed algorithms. Results demonstrate no error and a minimal error of 0.08 are achieved for the XOR and WBC classification tasks respectively, which are better than the original Booij's algorithm. The minimum number of epochs for XOR and Wisconsin breast cancer tasks are 35 and 26 respectively, which are also faster than the original Booij's algorithm - i.e. 135 (for XOR) and 97 (for WBC). Compared with the original algorithm with static momentum and learning rate, the proposed dynamic algorithms can control the convergence rate and learning performance more effectively.