Straggler Mitigation in Distributed Matrix Multiplication: Fundamental Limits and Optimal Coding

Qian Yu, Mohammad Ali Maddah-Ali, A. Salman Avestimehr

Research output: Contribution to journalArticlepeer-review

102 Scopus citations


We consider the problem of massive matrix multiplication, which underlies many data analytic applications, in a large-scale distributed system comprising a group of worker nodes. We target the stragglers' delay performance bottleneck, which is due to the unpredictable latency in waiting for slowest nodes (or stragglers) to finish their tasks. We propose a novel coding strategy, named entangled polynomial code, for designing the intermediate computations at the worker nodes in order to minimize the recovery threshold (i.e., the number of workers that we need to wait for in order to compute the final output). We demonstrate the optimality of entangled polynomial code in several cases, and show that it provides orderwise improvement over the conventional schemes for straggler mitigation. Furthermore, we characterize the optimal recovery threshold among all linear coding strategies within a factor of 2 using bilinear complexity, by developing an improved version of the entangled polynomial code. In particular, while evaluating bilinear complexity is a well-known challenging problem, we show that optimal recovery threshold for linear coding strategies can be approximated within a factor of 2 of this fundamental quantity. On the other hand, the improved version of the entangled polynomial code enables further and orderwise reduction in the recovery threshold, compared to its basic version. Finally, we show that the techniques developed in this paper can also be extended to several other problems such as coded convolution and fault-Tolerant computing, leading to tight characterizations.

Original languageEnglish (US)
Article number8949560
Pages (from-to)1920-1933
Number of pages14
JournalIEEE Transactions on Information Theory
Issue number3
StatePublished - Mar 2020
Externally publishedYes

Bibliographical note

Funding Information:
Manuscript received January 23, 2018; revised December 12, 2019; accepted December 12, 2019. Date of publication January 3, 2020; date of current version February 14, 2020. This work was supported in part by the Defense Advanced Research Projects Agency (DARPA) under Contract HR001117C0053, in part by the ONR under Award N000141612189, and in part by the NSF under Grant CCF-1703575 and Grant NeTS-1419632. This article was presented at the ISIT 2018.

Publisher Copyright:
© 1963-2012 IEEE.


  • Distributed computing
  • coded computing
  • matrix multiplication
  • straggler mitigation


Dive into the research topics of 'Straggler Mitigation in Distributed Matrix Multiplication: Fundamental Limits and Optimal Coding'. Together they form a unique fingerprint.

Cite this