Leveraging openstack and ceph for a controlled-access data cloud

Evan F. Bollig, Yectli A. Huerta, Graham T. Allan, Mathew Mix, Benjamin J. Lynch, Edward A. Munsell, Raychel M. Benson, Brent Swartz

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

While traditional HPC has and continues to satisfy most workows, a new generation of researchers has emerged looking for sophisticated, scalable, on-demand, and self-service control of compute infrastructure in a cloud-like environment. Many also seek safe harbors to operate on or store sensitive and/or controlled-access data in a high capacity environment. To cater to these modern users, the Minnesota Supercomputing Institute designed and deployed Stratus, a locally-hosted cloud environment powered by the OpenStack platform, and backed by Ceph storage. The subscription-based service complements existing HPC systems by satisfying the following unmet needs of our users: a) on-demand availability of compute resources; b) long-running jobs (i.e., > 30 days); c) container-based computing with Docker; and d) adequate security controls to comply with controlled-access data requirements. This document provides an in-depth look at the design of Stratus with respect to security and compliance with the NIH's controlled-access data policy. Emphasis is placed on lessons learned while integrating OpenStack and Ceph features into a so-called “walled garden”, and how those technologies inuenced the security design. Many features of Stratus, including tiered secure storage with the introduction of a controlled-access data “cache”, fault-tolerant live-migrations, and fully integrated two-factor authentication, depend on recent OpenStack and Ceph features.

Original languageEnglish (US)
Title of host publicationPractice and Experience in Advanced Research Computing 2018
Subtitle of host publicationSeamless Creativity, PEARC 2018
PublisherAssociation for Computing Machinery
ISBN (Print)9781450364461
DOIs
StatePublished - Jul 22 2018
Event2018 Practice and Experience in Advanced Research Computing Conference: Seamless Creativity, PEARC 2018 - Pittsburgh, United States
Duration: Jul 22 2017Jul 26 2017

Publication series

NameACM International Conference Proceeding Series

Other

Other2018 Practice and Experience in Advanced Research Computing Conference: Seamless Creativity, PEARC 2018
CountryUnited States
CityPittsburgh
Period7/22/177/26/17

Bibliographical note

Publisher Copyright:
© 2018 Copyright held by the owner/author(s).

Copyright:
Copyright 2018 Elsevier B.V., All rights reserved.

Keywords

  • Ceph
  • Cloud Computing
  • DbGaP
  • Docker
  • OpenStack
  • Private Cloud
  • Protected Data
  • S3

Fingerprint Dive into the research topics of 'Leveraging openstack and ceph for a controlled-access data cloud'. Together they form a unique fingerprint.

Cite this