Stochastic optimal control via forward and backward stochastic differential equations and importance sampling

摘要

In this paper we present a novel sampling-based numerical scheme designed tosolve a certain class of stochastic optimal control problems, utilizing forwardand backward stochastic differential equations (FBSDEs). By means of anonlinear version of the Feynman-Kac lemma, we obtain a probabilisticrepresentation of the solution to the nonlinear Hamilton-Jacobi-Bellmanequation, expressed in the form of a decoupled system of FBSDEs. This system ofFBSDEs can then be simulated by employing linear regression techniques. Toenhance the efficiency of the proposed scheme when treating more complexnonlinear systems, we then derive an iterative modification based on Girsanov'stheorem on the change of measure, which features importance sampling. Themodified scheme is capable of learning the optimal control without requiring aninitial guess. We present simulations that validate the algorithm anddemonstrate its efficiency in treating nonlinear dynamics.