Accelerating distributed workflows with edge resources

Siddharth Ramakrishnan, Robert Reutiman, Abhishek Chandra, Jon B Weissman

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Scopus citations

Abstract

Distributed data-intensive workflow applications are increasingly relying on and integrating remote resources including community data sources, services, and computational platforms. Increasingly, these are made available as data, SAAS, and IAAS clouds. The execution of distributed data-intensive workflow applications can exposé network bottlenecks between clouds that compromise performance. In this paper, we focus on alleviating network bottlenecks by using a proxy network. In particular, we show how proxies can eliminate network bottlenecks by smart routing and perform in-network computations to boost workflow application performance. A novel aspect of our work is the inclusion of multiple proxies to accelerate different workflow stages optimizing different performance metrics. We show that the approach is effective for workflow applications and broadly applicable. Using Montage1 as an exemplar workflow application, results obtained through experiments on Planet Lab showed how different proxies acting in a variety of roles can accelerate distinct stages of Montage. Our microbenchmarksalso show that routing data through select proxies can accelerate network transfer for TCP/UDP bandwidth, delay, and jitter, in general.

Original languageEnglish (US)
Title of host publicationProceedings - IEEE 27th International Parallel and Distributed Processing Symposium Workshops and PhD Forum, IPDPSW 2013
PublisherIEEE Computer Society
Pages2129-2138
Number of pages10
ISBN (Print)9780769549798
DOIs
StatePublished - Jan 1 2013
Event2013 IEEE 37th Annual Computer Software and Applications Conference, COMPSAC 2013 - Boston, MA, Japan
Duration: Jul 22 2013Jul 26 2013

Publication series

NameProceedings - IEEE 27th International Parallel and Distributed Processing Symposium Workshops and PhD Forum, IPDPSW 2013

Conference

Conference2013 IEEE 37th Annual Computer Software and Applications Conference, COMPSAC 2013
CountryJapan
CityBoston, MA
Period7/22/137/26/13

Keywords

  • Distributed computing
  • data-intensive computing
  • network systems
  • workflows

Fingerprint Dive into the research topics of 'Accelerating distributed workflows with edge resources'. Together they form a unique fingerprint.

  • Cite this

    Ramakrishnan, S., Reutiman, R., Chandra, A., & Weissman, J. B. (2013). Accelerating distributed workflows with edge resources. In Proceedings - IEEE 27th International Parallel and Distributed Processing Symposium Workshops and PhD Forum, IPDPSW 2013 (pp. 2129-2138). [6651119] (Proceedings - IEEE 27th International Parallel and Distributed Processing Symposium Workshops and PhD Forum, IPDPSW 2013). IEEE Computer Society. https://doi.org/10.1109/IPDPSW.2013.240