Sensitivity analysis using Ito-Malliavin calculus and martingales, and application to stochastic optimal control

摘要

We consider a multidimensional diffusion process (X-t(alpha)) 0 <= t <= T whose dynamics depends on a parameter alpha. Our first purpose is to write as an expectation the sensitivity del(alpha)J(alpha) for the expected cost J(alpha) = E(f(X-T(alpha))), in order to evaluate it using Monte Carlo simulations. This issue arises, for example, from stochastic control problems (where the controller is parameterized, which reduces the control problem to a parametric optimization one) or from model misspecifications in finance. Previous evaluations of del(alpha)J(alpha) using simulations were limited to smooth cost functions f or to diffusion coefficients not depending on alpha (see Yang and Kushner, SIAM J. Control Optim., 29 (1991), pp. 1216-1249). In this paper, we cover the general case, deriving three new approaches to evaluate del(alpha)J(alpha), which we call the Malliavin calculus approach, the adjoint approach, and the martingale approach. To accomplish this, we leverage Ito calculus, Malliavin calculus, and martingale arguments. In the second part of this work, we provide discretization procedures to simulate the relevant random variables; then we analyze their respective errors. This analysis proves that the discretization error is essentially linear with respect to the time step. This result, which was already known in some specific situations, appears to be true in this much wider context. Finally, we provide numerical experiments in random mechanics and finance and compare the different methods in terms of variance, complexity, computational time, and time discretization error.