Regularized ESN Based on TSVD

摘要

Echo state networks (ESNs) are a novel kind of recurrent neural networks., with the idea of using a large randomly and sparsely connected recurrent layer. Pseudoinverse is an effective method for training ESNs. However, the pseudoinverse method is liable to produce ill-posed problems. To avoid these problems, truncated singular value decomposition (TSVD) is introduced to calculate the output weights of ESNs. First, an oversize echo state network is used to map the inputs into a large reservoir. Then, the smaller singular values as well as the corresponding eigenvectors are removed. Finally, the output weights are calculated using the remainder singular values based on the pseudoinverse method. The experimental results on benchmark problems shows the effectiveness of the proposed method.