Echo in a small-world reservoir: Time-series prediction using an economical recurrent neural network

摘要

A small-world topology has been found in the cortical neural connectivity. However, the role of the topology in neural information processing has yet not been well understood. In this article, we investigate the performance of an echo state network (ESN) within a small-world topology in an economical or cost-effective environment, i.e., reduced number of input/output reservoir nodes. The ESN, a type of recurrent neural network, has a reservoir network where nodes are connected to each other with fixed weights. We introduce the small-world topology into the reservoir network. The ESN learns about the connected weights from the reservoir nodes to an output layer. In order to leverage the potential of the small-world topology, we limit the number of the reservoir nodes that receive external input (i.e., input nodes) or omit their signals to the output layer (i.e., output nodes). In addition, we segregate the input nodes from the output nodes, thereby necessitating the propagation of the input signals to the output nodes through the small-world reservoir. In our experiment, the ESNs learned to predict the next input of chaotic time-series. The small-world ESN exhibited high performance even when the number of input and output nodes was reduced, whereas the performance of the standard random or fully connected ESNs declined with reduced number of nodes.