This paper considers a class of constrained stochastic composite optimizationproblems whose objective function is given by the summation of a differentiable(possibly nonconvex) component, together with a certain non-differentiable (butconvex) component. In order to solve these problems, we propose a randomizedstochastic projected gradient (RSPG) algorithm, in which proper mini-batch ofsamples are taken at each iteration depending on the total budget of stochasticsamples allowed. The RSPG algorithm also employs a general distance function toallow taking advantage of the geometry of the feasible region. Complexity ofthis algorithm is established in a unified setting, which shows nearly optimalcomplexity of the algorithm for convex stochastic programming. Apost-optimization phase is also proposed to significantly reduce the varianceof the solutions returned by the algorithm. In addition, based on the RSPGalgorithm, a stochastic gradient free algorithm, which only uses the stochasticzeroth-order information, has been also discussed. Some preliminary numericalresults are also provided.
展开▼
