HACCS: Heterogeneity-Aware Clustered Client Selection for Accelerated Federated Learning

Joel Wolfrath, Nikhil Sreekumar, Dhruv Kumar, Yuanli Wang, Abhishek Chandra

Research output: Chapter in Book/Report/Conference proceedingConference contribution

12 Scopus citations

Abstract

Federated Learning is a machine learning paradigm where a global model is trained in-situ across a large number of distributed edge devices. While this technique avoids the cost of transferring data to a central location and achieves a strong degree of privacy, it presents additional challenges due to the heterogeneous hardware resources available for training. Furthermore, data is not independent and identically distributed (IID) across all edge devices, resulting in statistical heterogeneity across devices. Due to these constraints, client selection strategies play an important role for timely convergence during model training. Existing strategies ensure that each individual device is included, at least periodically, in the training process. In this work, we propose HACCS, a Heterogeneity-Aware Clustered Client Selection system that identifies and exploits the statistical heterogeneity by representing all distinguishable data distributions instead of individual devices in the training process. HACCS is robust to individual device dropout, provided other devices in the system have similar data distributions. We propose privacy-preserving methods for estimating these client distributions and clustering them. We also propose strategies for leveraging these clusters to make scheduling decisions in a federated learning system. Our evaluation on real-world datasets suggests that our framework can provide 18% -38% reduction in time to convergence compared to the state of the art without any compromise in accuracy.

Original languageEnglish (US)
Title of host publicationProceedings - 2022 IEEE 36th International Parallel and Distributed Processing Symposium, IPDPS 2022
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages985-995
Number of pages11
ISBN (Electronic)9781665481069
DOIs
StatePublished - 2022
Event36th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2022 - Virtual, Online, France
Duration: May 30 2022Jun 3 2022

Publication series

NameProceedings - 2022 IEEE 36th International Parallel and Distributed Processing Symposium, IPDPS 2022

Conference

Conference36th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2022
Country/TerritoryFrance
CityVirtual, Online
Period5/30/226/3/22

Bibliographical note

Funding Information:
ACKNOWLEDGEMENT This research was supported in part by the NSF under grant CNS-1717834.

Publisher Copyright:
© 2022 IEEE.

Keywords

  • Clustering
  • Federated Learning
  • Non-IID data
  • Scheduling

Fingerprint

Dive into the research topics of 'HACCS: Heterogeneity-Aware Clustered Client Selection for Accelerated Federated Learning'. Together they form a unique fingerprint.

Cite this