Echo state network-based Q-learning method for optimal battery control of offices combined with renewable energy

摘要

An echo state network (ESN)-based Q-learning method is developed for optimal energy management of an office, where the solar energy is introduced as the renewable source, and a battery is installed with a control unit. The energy consumption in the office, also considered as the energy demand, is separated into those from sockets, lights and air-conditioners. First, ESNs, well known for their excellent modelling performance for time series, are employed to model the time series of the real-time electricity rate, renewable energy and energy demand as periodic functions. Second, given the periodic models of the electricity rate, renewable energy and energy demand, an ESN-based Q-learning method with the Q-function approximated by an ESN is developed and implemented to determine the optimal charging/discharging/idle strategies for the battery in the office, so that the total cost of electricity from the grid can be reduced. Finally, numerical analysis is conducted to illustrate the performance of the developed method.