Data-driven dynamic multi-objective optimal control: A Hamiltonian-inequality driven satisficing reinforcement learning approach

摘要

This paper presents an iterative data-driven algorithm for solving dynamic multi-objective (MO) optimal control problems arising in control of nonlinear continuous-time systems with multiple objectives. It is first shown that the Hamiltonian function corresponding to each objective can serve as a comparison function to compare the performance of admissible policies. Relaxed Hamilton-Jacobi-bellman (HJB) equations in terms of HJB inequalities are then solved in a dynamic constrained MO framework to find Pareto-optimal solutions. Relation to satisficing (good enough) decision-making framework is shown. A Sum-of-Square (SOS)-based iterative algorithm is developed to solve the formulated MO optimization with HJB inequalities. To obviate the requirement of complete knowledge of the system dynamics, a data-driven satisficing reinforcement learning approach is proposed to solve the SOS optimization problem in real-time using only the information of the system trajectories measured during a time interval without having full knowledge of the system dynamics. Finally, a simulation example is provided to show the effectiveness of the proposed algorithm.