A primal-dual type algorithm with the O(1/t) convergence rate for large scale constrained convex programs

摘要

This paper considers large scale constrained convex programs. These are often difficult to solve by interior point methods or other Newton-type methods due to the prohibitive computation and storage complexity for Hessians or matrix inversions. Instead, large scale constrained convex programs are often solved by gradient based methods or decomposition based methods. The conventional primal-dual subgradient method, also known as the Arrow-Hurwicz-Uzawa subgradient method, is a low complexity algorithm with the O(1/√t) convergence rate, where t is the number of iterations. If the objective and constraint functions are separable, the Lagrangian dual type method can decompose a large scale convex program into multiple parallel small scale convex programs. The classical dual gradient algorithm is an example of Lagrangian dual type methods and has convergence rate O(1/√t). Recently, the authors of the current paper proposed a new Lagrangian dual type algorithm with faster O(1/t) convergence. However, if the objective or constraint functions are not separable, each iteration requires to solve a large scale unconstrained convex program, which can have huge complexity. This paper proposes a new primal-dual type algorithm, which only involves simple gradient updates at each iteration and has O(1/t) convergence.