Designing algorithms for an optimization model often amounts to maintaining a balance between the degree of information to request from the model on the one hand, and the computational speed to expect on the other hand. Naturally, the more information is available, the faster one can expect the algorithm to converge. The popular algorithm of ADMM demands that objective function is easy to optimize once the coupled constraints are shifted to the objective with multipliers. However, in many applications this assumption does not hold; instead, often only some noisy estimations of the gradient of the objective—or even only the objective itself—are available. This paper aims to bridge this gap. We present a suite of variants of the ADMM, where the trade-offs between the required information on the objective and the computational complexity are explicitly given. The new variants allow the method to be applicable on a much broader class of problems where only noisy estimations of the gradient or the function values are accessible, yet the flexibility is achieved without sacrificing the computational complexity bounds.
Bibliographical noteFunding Information:
Bo Jiang: Research of this author was supported in part by National Natural Science Foundation of China (Grant Nos. 11401364 and 11771269) and Program for Innovative Research Team of Shanghai University of Finance and Economics. Shuzhong Zhang: Research of this author was supported in part by National Science Foundation (Grant CMMI-1462408).
- Alternating direction method of multipliers (ADMM)
- Direct method
- First-order method
- Iteration complexity
- Stochastic approximation