Iterative Dynamic Programming - From Curiosity to a Practical Optimization Procedure

摘要

Dynamic programming, developed by Richard Bellman in 1957, is a very attractive optimization procedure conceptually, but attempts to use it for other than very simple optimal control problems have not been successful. The difficulties arise from the need for interpolation, the "curse of dimensionality", and the requirement of a large number of grid points to get a reasonably accurate solution. Several systematic developments had to be made to make dynamic programming suitable for optimal control problems. These developments, as outlined here, led to iterative dynamic programming which can now easily handle optimal control problems of some systems consisting of more than 300 differential equations and more than 300 control variables. Computations with IDP for realistic engineering problems can be done sufficiently fast, so that we may use IDP to solve the optimal control problem within a sampling period and provide in essence a feed-back control policy.