Modelling a Penicillin Fermentation Process Using Attention-Based Echo State Networks Optimized by Covariance Matrix Adaption Evolutionary Strategy

摘要

Echo state network(ESN)has emerged as an effective alternative to conventional recurrent neural networks due to its simple training process and good modelling ability for solving a variety of problems,especially time-series modelling tasks.To improve modelling capability and to decrease the reservoir topology complexity,a new attention mechanism based ESN optimised by covariance matrix adaption evolutionary strategy(CMA-ES)is proposed in this paper.CMA-ES is a stochastic and derivative-free algorithm for solving non-1inear optimization problems.Attention mechanism is incorporated to guide ESN to focus on regions of interest relevant to the modelling task.The proposed optimised ESN with attention mechanism is used to model a fed-batch penicillin fermentation process and the results are better than those from the standard ESN and ESN with attention mechanism.