TY - GEN
T1 - Provably communication-efficient asynchronous distributed inference for convex and nonconvex problems
AU - Ren, Jineng
AU - Haupt, Jarvis
PY - 2019/2/20
Y1 - 2019/2/20
N2 - This paper proposes and analyzes an asynchronous communication-efficient distributed optimization framework for a general type of machine learning and signal processing problems. At each iteration, worker machines compute gradients of a known empirical loss function using their own local data, and a master machine solves a related minimization problem to update the current estimate. We establish that the proposed algorithm converges with a sublinear rate over the number of communication rounds, coinciding with the best theoretical rate that can be achieved for nonconvex nonsmooth problems. Moreover, under a strong convexity assumption of the smooth part of the loss function, linear convergence is established. Extensive numerical experiments show that the performance of the proposed approach indeed improves - sometimes significantly - over other state-of-the-art algorithms in terms of total communication efficiency.
AB - This paper proposes and analyzes an asynchronous communication-efficient distributed optimization framework for a general type of machine learning and signal processing problems. At each iteration, worker machines compute gradients of a known empirical loss function using their own local data, and a master machine solves a related minimization problem to update the current estimate. We establish that the proposed algorithm converges with a sublinear rate over the number of communication rounds, coinciding with the best theoretical rate that can be achieved for nonconvex nonsmooth problems. Moreover, under a strong convexity assumption of the smooth part of the loss function, linear convergence is established. Extensive numerical experiments show that the performance of the proposed approach indeed improves - sometimes significantly - over other state-of-the-art algorithms in terms of total communication efficiency.
KW - Asynchronous
KW - Communication-efficient
KW - Convergence
KW - Distributed algorithm
KW - Nonconvex
UR - http://www.scopus.com/inward/record.url?scp=85063072786&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85063072786&partnerID=8YFLogxK
U2 - 10.1109/GlobalSIP.2018.8646525
DO - 10.1109/GlobalSIP.2018.8646525
M3 - Conference contribution
AN - SCOPUS:85063072786
T3 - 2018 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2018 - Proceedings
SP - 638
EP - 642
BT - 2018 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2018 - Proceedings
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2018 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2018
Y2 - 26 November 2018 through 29 November 2018
ER -