Optimal Controls That Maximize the Probability of Being in a Given Set at a Given Time

摘要

An optimal control problem is considered for a class of nonlinear controlled stochastic systems, the states of which are r(m) valued Markov processes. Suppose that an open bounded set d contained in r(m), a subset a contained in d and a time t is greater than 0 are given. The problem is treated of maximizing the probability that the state of the system will be in the set A at the time T after having remained in D throughout the interval 0,T. Sufficient conditions on optimal controls, of a dynamic programming type, are derived. These conditions require the existence of a smooth solution to a nonlinear partial integro-differential equation. A stochastic second order system, describing a perturbed sine wave oscillator, is used as a test case, and a numerical method for computing optimal controls is suggested. The performance of the oscillator for both optimal and optimal control laws, is investigated by computing the probability that the state of the oscillator will be in a given neighborhood of its limit cycle, and by computing the expected value of the time it is there.