Accelerating Distributed Deep Learning using Multi-Path RDMA in Data Center Networks

Feng Tian, Yang Zhang, Wei Ye, Cheng Jin, Ziyan Wu, Zhi Li Zhang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Data center networks (DCNs) have widely deployed RDMA to support data-intensive applications such as machine learning. While DCNs are designed with rich multi-path topology, current RDMA (hardware) technology does not support multi-path transport. In this paper we advance Maestro- a purely software-basedmulti-path RDMA solution - to effectively utilize the rich multi-path topology for load balancing and reliability. As a "middleware"operating at the user-space, Maestro is modulaR@and software-defined:Maestro decouples path selection and load balancing mechanisms from hardware features, and allows DCN operators and applications to make flexible decisions by employing the best mechanisms as needed. As such, Maestro can be readily deployed using existing RDMA hardware (NICs) to support distributed deep learning (DDL) applications. Our experiments show that Maestro is capable of fully utilizing multiple paths with negligible CPU overheads, thereby enhancing the performance of DDL applications.

Original languageEnglish (US)
Title of host publicationSOSR 2021 - Proceedings of the 2021 ACM SIGCOMM Symposium on SDN Research
PublisherAssociation for Computing Machinery, Inc
Pages88-100
Number of pages13
ISBN (Electronic)9781450390842
DOIs
StatePublished - Oct 11 2021
Event2021 ACM SIGCOMM Symposium on SDN Research, SOSR 2021 - Virtual, Online, United States
Duration: Sep 20 2021Sep 21 2021

Publication series

NameProceedings of the ACM SIGCOMM Symposium on SDN Research (SOSR)

Conference

Conference2021 ACM SIGCOMM Symposium on SDN Research, SOSR 2021
Country/TerritoryUnited States
CityVirtual, Online
Period9/20/219/21/21

Bibliographical note

Funding Information:
The research was supported in part by NSF under Grants CNS-1618339, CNS-1814322, CNS-1831140, CNS-1836772, CNS-1901103, CNS-2106771 and CCF-2123987.

Publisher Copyright:
© 2021 ACM.

Keywords

  • Data Center Networks
  • Distributed Deep Learning
  • Multi-Path RDMA

Fingerprint

Dive into the research topics of 'Accelerating Distributed Deep Learning using Multi-Path RDMA in Data Center Networks'. Together they form a unique fingerprint.

Cite this