Adaptive Bit Allocation for Communication-Efficient Distributed Optimization

Hadi Reisizadeh; Behrouz Touri; Soheil Mohajer

doi:10.1109/CDC45484.2021.9683441

Adaptive Bit Allocation for Communication-Efficient Distributed Optimization

Hadi Reisizadeh
, Behrouz Touri
, Soheil Mohajer

Electrical and Computer Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

2 Scopus citations

Abstract

We propose an adaptive quantization method for two important distributed computation tasks: federated learning and distributed optimization. In both settings, we propose adaptive bit allocation schemes that allow nodes to trade their bandwidth with a minimal communication overhead. We show that the proposed schemes lead to an improvement in the speed of convergence of these methods compared to a uniform bit allocation method, especially when the data distribution among the nodes is skewed. Our theoretical results are corroborated by extensive simulations on various datasets.

Original language	English (US)
Title of host publication	60th IEEE Conference on Decision and Control, CDC 2021
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	1994-2001
Number of pages	8
ISBN (Electronic)	9781665436595
DOIs	https://doi.org/10.1109/CDC45484.2021.9683441
State	Published - 2021
Event	60th IEEE Conference on Decision and Control, CDC 2021 - Austin, United States Duration: Dec 13 2021 → Dec 17 2021

Publication series

Name	Proceedings of the IEEE Conference on Decision and Control
Volume	2021-December
ISSN (Print)	0743-1546
ISSN (Electronic)	2576-2370

Conference

Conference	60th IEEE Conference on Decision and Control, CDC 2021
Country/Territory	United States
City	Austin
Period	12/13/21 → 12/17/21

Bibliographical note

Funding Information:
The work of H. Reisizadeh and S. Mohajer is supported in part by the National Science Foundation under Grants CCF-1617884.

Publisher Copyright:
© 2021 IEEE.

Publisher link

10.1109/CDC45484.2021.9683441

Find It

Find It at U of M Libraries

Cite this

Reisizadeh, H., Touri, B., & Mohajer, S. (2021). Adaptive Bit Allocation for Communication-Efficient Distributed Optimization. In 60th IEEE Conference on Decision and Control, CDC 2021 (pp. 1994-2001). (Proceedings of the IEEE Conference on Decision and Control; Vol. 2021-December). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CDC45484.2021.9683441

Adaptive Bit Allocation for Communication-Efficient Distributed Optimization. / Reisizadeh, Hadi; Touri, Behrouz; Mohajer, Soheil.
60th IEEE Conference on Decision and Control, CDC 2021. Institute of Electrical and Electronics Engineers Inc., 2021. p. 1994-2001 (Proceedings of the IEEE Conference on Decision and Control; Vol. 2021-December).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Reisizadeh, H, Touri, B & Mohajer, S 2021, Adaptive Bit Allocation for Communication-Efficient Distributed Optimization. in 60th IEEE Conference on Decision and Control, CDC 2021. Proceedings of the IEEE Conference on Decision and Control, vol. 2021-December, Institute of Electrical and Electronics Engineers Inc., pp. 1994-2001, 60th IEEE Conference on Decision and Control, CDC 2021, Austin, United States, 12/13/21. https://doi.org/10.1109/CDC45484.2021.9683441

@inproceedings{ba5a33e9a261466ca8e58368d9f2acc2,

title = "Adaptive Bit Allocation for Communication-Efficient Distributed Optimization",

abstract = "We propose an adaptive quantization method for two important distributed computation tasks: federated learning and distributed optimization. In both settings, we propose adaptive bit allocation schemes that allow nodes to trade their bandwidth with a minimal communication overhead. We show that the proposed schemes lead to an improvement in the speed of convergence of these methods compared to a uniform bit allocation method, especially when the data distribution among the nodes is skewed. Our theoretical results are corroborated by extensive simulations on various datasets.",

author = "Hadi Reisizadeh and Behrouz Touri and Soheil Mohajer",

note = "Funding Information: The work of H. Reisizadeh and S. Mohajer is supported in part by the National Science Foundation under Grants CCF-1617884. Publisher Copyright: {\textcopyright} 2021 IEEE.; 60th IEEE Conference on Decision and Control, CDC 2021 ; Conference date: 13-12-2021 Through 17-12-2021",

year = "2021",

doi = "10.1109/CDC45484.2021.9683441",

language = "English (US)",

series = "Proceedings of the IEEE Conference on Decision and Control",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "1994--2001",

booktitle = "60th IEEE Conference on Decision and Control, CDC 2021",

}

TY - GEN

T1 - Adaptive Bit Allocation for Communication-Efficient Distributed Optimization

AU - Reisizadeh, Hadi

AU - Touri, Behrouz

AU - Mohajer, Soheil

PY - 2021

Y1 - 2021

N2 - We propose an adaptive quantization method for two important distributed computation tasks: federated learning and distributed optimization. In both settings, we propose adaptive bit allocation schemes that allow nodes to trade their bandwidth with a minimal communication overhead. We show that the proposed schemes lead to an improvement in the speed of convergence of these methods compared to a uniform bit allocation method, especially when the data distribution among the nodes is skewed. Our theoretical results are corroborated by extensive simulations on various datasets.

AB - We propose an adaptive quantization method for two important distributed computation tasks: federated learning and distributed optimization. In both settings, we propose adaptive bit allocation schemes that allow nodes to trade their bandwidth with a minimal communication overhead. We show that the proposed schemes lead to an improvement in the speed of convergence of these methods compared to a uniform bit allocation method, especially when the data distribution among the nodes is skewed. Our theoretical results are corroborated by extensive simulations on various datasets.

UR - https://www.scopus.com/pages/publications/85126038764

UR - https://www.scopus.com/pages/publications/85126038764#tab=citedBy

U2 - 10.1109/CDC45484.2021.9683441

DO - 10.1109/CDC45484.2021.9683441

M3 - Conference contribution

AN - SCOPUS:85126038764

T3 - Proceedings of the IEEE Conference on Decision and Control

SP - 1994

EP - 2001

BT - 60th IEEE Conference on Decision and Control, CDC 2021

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 60th IEEE Conference on Decision and Control, CDC 2021

Y2 - 13 December 2021 through 17 December 2021

ER -

Adaptive Bit Allocation for Communication-Efficient Distributed Optimization

Abstract

Publication series

Conference

Bibliographical note

Publisher link

Find It

Other files and links

Fingerprint

Cite this