Performance improvement of backpropagation algorithm by automatic activation function gain tuning using fuzzy logic

摘要

We propose a method for improving the performance of the backpropagation algorithm using a fuzzy logic system for automatically tuning the activation function gain. Instead of a fixed activation function gain, the fuzzy logic system is used to dynamically adjust the gain, based upon a set of problem domain heuristics derived from a preliminary simulation study. The efficacy of the proposed method is verified by means of simulations on a parity problem, a function approximation problem, and a pattern recognition problem. The results show that the proposed method improves considerably on the performance of the general backpropagation algorithm, including when using momentum.