Computations of Descent Directions in Stochastic Optimization Problems with Invariant Distributions

摘要

One of the main problems in stochastic optimization is the minimization of mean value functions F(x)=Eu(A(omega)x-b(omega)) s.t. x is a member of D, where (A(omega),b(omega)) is a random matrix, u is a loss function on R(m) and D is a convex subset of R(n). If the distribution mu of (A(omega),b(omega)) is invariant with respect to an affine transformation lambda of R(n)(m+1), i.e., if lambda(mu)=mu, and if for a given R(n) there exists a vector y=y(x) such that lambda(A,b)(sup x sub-1)=Ay-b a.s., then F(y)=F(x) and, consequently, d=y-x is a descent direction of F at x, provided that F is not constant on the line segment joining x and y. For many practically important distributions mu the affine transformation lambda, a vextor y(x) and therefore also a direction of decrease d=y(x)-x is constructed without using any derivatives of F.