On the linear convergence of the alternating direction method of multipliers

Mingyi Hong, Zhi Quan Luo

Research output: Contribution to journalArticlepeer-review

246 Scopus citations

Abstract

We analyze the convergence rate of the alternating direction method of multipliers (ADMM) for minimizing the sum of two or more nonsmooth convex separable functions subject to linear constraints. Previous analysis of the ADMM typically assumes that the objective function is the sum of only two convex functions defined on two separable blocks of variables even though the algorithm works well in numerical experiments for three or more blocks. Moreover, there has been no rate of convergence analysis for the ADMM without strong convexity in the objective function. In this paper we establish the global R-linear convergence of the ADMM for minimizing the sum of any number of convex separable functions, assuming that a certain error bound condition holds true and the dual stepsize is sufficiently small. Such an error bound condition is satisfied for example when the feasible set is a compact polyhedron and the objective function consists of a smooth strictly convex function composed with a linear mapping, and a nonsmooth ℓ1 regularizer. This result implies the linear convergence of the ADMM for contemporary applications such as LASSO without assuming strong convexity of the objective function.

Original languageEnglish (US)
Pages (from-to)165-199
Number of pages35
JournalMathematical Programming
Volume162
Issue number1-2
DOIs
StatePublished - Mar 1 2017

Bibliographical note

Publisher Copyright:
© 2016, Springer-Verlag Berlin Heidelberg and Mathematical Optimization Society.

Keywords

  • Alternating directions of multipliers
  • Dual ascent
  • Error bound
  • Linear convergence

Fingerprint

Dive into the research topics of 'On the linear convergence of the alternating direction method of multipliers'. Together they form a unique fingerprint.

Cite this