Jupyter as Common Technology Platform for Interactive HPC Services

Research output: Contribution to conferencePaperpeer-review

3 Scopus citations

Abstract

The Minnesota Supercomputing Institute has implemented Jupyterhub and the Jupyter notebook server as a general-purpose point-of-entry to interactive high performance computing services. This mode of operation runs counter to traditional job-oriented HPC operations, but offers significant advantages for ease-of-use, data exploration, prototyping, and workflow development. From the user perspective, these features bring the computing cluster nearer to parity with emerging cloud computing options. On the other hand, retreating from fully-scheduled, job-based resource allocation poses challenges for resource availability and utilization efficiency, and can involve tools and technologies outside the typical core competencies of a supercomputing center's operations staff. MSI has attempted to mitigate these challenges by adopting Jupyter as a common technology platform for interactive services, capable of providing command-line, graphical, and workflow-oriented access to HPC resources while still integrating with job scheduling systems and using existing compute resources. This paper will describe the mechanisms that MSI has put in place, advantages for research and instructional uses, and lessons learned.
Original languageEnglish (US)
Number of pages6
DOIs
StatePublished - Jul 26 2018
Event2018 Practice and Experience in Advanced Research Computing Conference: Seamless Creativity, PEARC 2018 - Pittsburgh, United States
Duration: Jul 22 2017Jul 26 2017

Other

Other2018 Practice and Experience in Advanced Research Computing Conference: Seamless Creativity, PEARC 2018
CountryUnited States
CityPittsburgh
Period7/22/177/26/17

Bibliographical note

6 pages, 2 figures, in PEARC '18: Proceedings of Practice and Experience in Advanced Research Computing, July 22--26, 2018, Pittsburgh, PA, USA

Keywords

  • cs.DC
  • cs.CY

Fingerprint Dive into the research topics of 'Jupyter as Common Technology Platform for Interactive HPC Services'. Together they form a unique fingerprint.

Cite this