Data-Driven Model-Free Model-Reference Nonlinear Virtual State Feedback Control from Input-Output Data

摘要

In this paper we show that it is possible to learn high performance model reference controllers using a model-free Q-learning approach, based on input-output (IO) samples collected from the controlled process. Under observability assumptions for the process, virtual extended state-space process models of different orders are built from the IO collected samples. We prove that state-space control of these virtual processes is equivalent to IO control of the initial process. For the virtual state-space process, high performance nonlinear Neural Network (NN) state-feedback controllers are learned based on the IO data collected from the initial process, to achieve output model-reference tracking control. Control learning is a two-step model-free process: an IO model-free controller first drives the process exploration of a wide operating range for IO samples collection that are then used to model-free learn the NN controllers. The approach is successfully validated on a highly nonlinear coupled aerodynamic system.