Adaptive Bit Allocation for Communication-Efficient Distributed Optimization

Hadi Reisizadeh, Behrouz Touri, Soheil Mohajer

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

We propose an adaptive quantization method for two important distributed computation tasks: federated learning and distributed optimization. In both settings, we propose adaptive bit allocation schemes that allow nodes to trade their bandwidth with a minimal communication overhead. We show that the proposed schemes lead to an improvement in the speed of convergence of these methods compared to a uniform bit allocation method, especially when the data distribution among the nodes is skewed. Our theoretical results are corroborated by extensive simulations on various datasets.

Original languageEnglish (US)
Title of host publication60th IEEE Conference on Decision and Control, CDC 2021
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1994-2001
Number of pages8
ISBN (Electronic)9781665436595
DOIs
StatePublished - 2021
Event60th IEEE Conference on Decision and Control, CDC 2021 - Austin, United States
Duration: Dec 13 2021Dec 17 2021

Publication series

NameProceedings of the IEEE Conference on Decision and Control
Volume2021-December
ISSN (Print)0743-1546
ISSN (Electronic)2576-2370

Conference

Conference60th IEEE Conference on Decision and Control, CDC 2021
Country/TerritoryUnited States
CityAustin
Period12/13/2112/17/21

Bibliographical note

Funding Information:
The work of H. Reisizadeh and S. Mohajer is supported in part by the National Science Foundation under Grants CCF-1617884.

Publisher Copyright:
© 2021 IEEE.

Fingerprint

Dive into the research topics of 'Adaptive Bit Allocation for Communication-Efficient Distributed Optimization'. Together they form a unique fingerprint.

Cite this