Approximate dynamic programming solutions with a single network adaptive critic for a class of nonlinear systems.

摘要

Approximate dynamic programming formulation implemented with an Adaptive Critic (AC) based neural network (NN) structure has evolved as a powerful technique for solving the Hamilton-Jacobi-Bellman (HJB) equations. As interest in ADP and the AC solutions are escalating with time, there is a dire need to consider possible enabling factors for their implementations. A typical AC structure consists of two interacting NNs which is computationally expensive. In this work, a new architecture, called the "Cost Function Based Single Network Adaptive Critic (J-SNAC)" is presented that eliminates one of the networks in a typical AC structure. This approach is applicable to a wide class of nonlinear systems in engineering. In the first paper, two problems have been solved with the AC and the J-SNAC approaches. Results are presented that show savings of about 50% of the computational costs by J-SNAC while having the same accuracy levels of the dual network structure in solving for optimal control. In the second paper, the plant dynamics with parametric uncertainties or unmodeled nonlinearities has been considered. The author discusses the dynamic rc-optimization of the J-SNAC controller that is used to capture the uncertainty but is not considered in the system model used for controller design. In the third paper, a non-quadratic cost function is used to incorporate control constraints. Necessary equations for optimal control arc derived and au algorithm is presented to solve the constrained-control problem with J-SNAC. The fourth paper presents a new controller design technique for a class of nonlinear impulse driven systems.