The development of efficient algorithms, based on optimization techniques, for learning the weights and architecture of recurrent neural networks (RNN), has recently, received much attention. Though many training methods have been successfully used in training RNNs, they suffer from certain fundamental drawbacks. Firstly, it is difficult to devise efficient learning algorithms that guarantee stability of the overall system. Secondly, they rely on some type of approximation for computing the partial derivative, which may be inadequate to train them for tasks involving long-term dependencies. Furthermore, no analytical results concerning the convergence and stability of these schemes have been obtained. In this paper we propose a recurrent bilinear perceptron (RBP) for time series prediction. Evolutionary programming, a multi-agent stochastic search technique, is used to optimally determine the model order and coefficients of the recurrent bilinear perceptrons. Unlike conventional techniques, where the model order is chosen first, and then the parameters of the model are determined, both the model order and the parameters are evolved simultaneously. The minimum description length (MDL) information criterion is used to evalaute each recurrent bilinear perceptron structure as a candidate solution. Experimental results on time series prediction indicate that RBPs are capable of performing better than simple bilinear models in terms of reduced normalized mean squared error (NMSE) and lower evolved model order.
展开▼
