Building a scalable forward flux sampling framework using big data and H PC

Ryan S. DeFever; Walter Hanger; Sapna Sarupria; Jon Kilgannon; Amy W. Apon; Linh B. Ngo

doi:10.1145/3332186.3332205

Building a scalable forward flux sampling framework using big data and H PC

Ryan S. DeFever
, Walter Hanger
, Sapna Sarupria
, Jon Kilgannon
, Amy W. Apon
, Linh B. Ngo

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

2 Scopus citations

Abstract

Forward flux sampling (FFS) is an established scientific method for sampling rare events in molecular simulations. However, as the difficulty of the scientific problem increases, the amount of data and the number of tasks required for FFS is challenging to manage with traditional scripting tools and languages for high performance computing. The SAFFIRE software framework has been developed to address these challenges. SAFFIRE utilizes Hadoop to manage a large number of tasks and data for large scale FFS simulations. The framework is shown to be highly scalable and able to support large scale FFS simulations. This enables studies of rare events in complex molecular systems on commodity cluster computing systems.

Original language	English (US)
Title of host publication	Proceedings of the Practice and Experience in Advanced Research Computing
Subtitle of host publication	Rise of the Machines (Learning), PEARC 2019
Publisher	Association for Computing Machinery
ISBN (Electronic)	9781450372275
DOIs	https://doi.org/10.1145/3332186.3332205
State	Published - Jul 28 2019
Externally published	Yes
Event	2019 Conference on Practice and Experience in Advanced Research Computing: Rise of the Machines (Learning), PEARC 2019 - Chicago, United States Duration: Jul 28 2019 → Aug 1 2019

Publication series

Name	ACM International Conference Proceeding Series

Conference

Conference	2019 Conference on Practice and Experience in Advanced Research Computing: Rise of the Machines (Learning), PEARC 2019
Country/Territory	United States
City	Chicago
Period	7/28/19 → 8/1/19

Bibliographical note

Funding Information:
RSD and SS acknowledge the support by the U.S. Department of Energy, Office of Science, Office of Basic Energy Sciences, under Award No. DE-SC0015448. AA and LBN acknowledge the support by the National Science Foundation under Award No. 1405767. Simulations were performed on the Palmetto Supercomputer Cluster of Clemson University and the Bridges Supercomputer through XSEDE.

Publisher Copyright:
© 2019 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM

Keywords

Data-intensive computing
Forward Flux Sampling
Hadoop
Molecular simulations
Rare events

Publisher link

10.1145/3332186.3332205

Find It

Find It at U of M Libraries

Cite this

DeFever, R. S., Hanger, W., Sarupria, S., Kilgannon, J., Apon, A. W., & Ngo, L. B. (2019). Building a scalable forward flux sampling framework using big data and H PC. In Proceedings of the Practice and Experience in Advanced Research Computing: Rise of the Machines (Learning), PEARC 2019 Article 3332205 (ACM International Conference Proceeding Series). Association for Computing Machinery. https://doi.org/10.1145/3332186.3332205

Building a scalable forward flux sampling framework using big data and H PC. / DeFever, Ryan S.; Hanger, Walter; Sarupria, Sapna et al.
Proceedings of the Practice and Experience in Advanced Research Computing: Rise of the Machines (Learning), PEARC 2019. Association for Computing Machinery, 2019. 3332205 (ACM International Conference Proceeding Series).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

DeFever, RS, Hanger, W, Sarupria, S, Kilgannon, J, Apon, AW & Ngo, LB 2019, Building a scalable forward flux sampling framework using big data and H PC. in Proceedings of the Practice and Experience in Advanced Research Computing: Rise of the Machines (Learning), PEARC 2019., 3332205, ACM International Conference Proceeding Series, Association for Computing Machinery, 2019 Conference on Practice and Experience in Advanced Research Computing: Rise of the Machines (Learning), PEARC 2019, Chicago, United States, 7/28/19. https://doi.org/10.1145/3332186.3332205

DeFever RS, Hanger W, Sarupria S, Kilgannon J, Apon AW, Ngo LB. Building a scalable forward flux sampling framework using big data and H PC. In Proceedings of the Practice and Experience in Advanced Research Computing: Rise of the Machines (Learning), PEARC 2019. Association for Computing Machinery. 2019. 3332205. (ACM International Conference Proceeding Series). doi: 10.1145/3332186.3332205

@inproceedings{6160e425252243f697eb20260fc146de,

title = "Building a scalable forward flux sampling framework using big data and H PC",

abstract = "Forward flux sampling (FFS) is an established scientific method for sampling rare events in molecular simulations. However, as the difficulty of the scientific problem increases, the amount of data and the number of tasks required for FFS is challenging to manage with traditional scripting tools and languages for high performance computing. The SAFFIRE software framework has been developed to address these challenges. SAFFIRE utilizes Hadoop to manage a large number of tasks and data for large scale FFS simulations. The framework is shown to be highly scalable and able to support large scale FFS simulations. This enables studies of rare events in complex molecular systems on commodity cluster computing systems.",

keywords = "Data-intensive computing, Forward Flux Sampling, Hadoop, Molecular simulations, Rare events",

author = "DeFever, \{Ryan S.\} and Walter Hanger and Sapna Sarupria and Jon Kilgannon and Apon, \{Amy W.\} and Ngo, \{Linh B.\}",

note = "Funding Information: RSD and SS acknowledge the support by the U.S. Department of Energy, Office of Science, Office of Basic Energy Sciences, under Award No. DE-SC0015448. AA and LBN acknowledge the support by the National Science Foundation under Award No. 1405767. Simulations were performed on the Palmetto Supercomputer Cluster of Clemson University and the Bridges Supercomputer through XSEDE. Publisher Copyright: {\textcopyright} 2019 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM; 2019 Conference on Practice and Experience in Advanced Research Computing: Rise of the Machines (Learning), PEARC 2019 ; Conference date: 28-07-2019 Through 01-08-2019",

year = "2019",

month = jul,

day = "28",

doi = "10.1145/3332186.3332205",

language = "English (US)",

series = "ACM International Conference Proceeding Series",

publisher = "Association for Computing Machinery",

booktitle = "Proceedings of the Practice and Experience in Advanced Research Computing",

}

TY - GEN

T1 - Building a scalable forward flux sampling framework using big data and H PC

AU - DeFever, Ryan S.

AU - Hanger, Walter

AU - Sarupria, Sapna

AU - Kilgannon, Jon

AU - Apon, Amy W.

AU - Ngo, Linh B.

N1 - Funding Information: RSD and SS acknowledge the support by the U.S. Department of Energy, Office of Science, Office of Basic Energy Sciences, under Award No. DE-SC0015448. AA and LBN acknowledge the support by the National Science Foundation under Award No. 1405767. Simulations were performed on the Palmetto Supercomputer Cluster of Clemson University and the Bridges Supercomputer through XSEDE. Publisher Copyright: © 2019 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM

PY - 2019/7/28

Y1 - 2019/7/28

N2 - Forward flux sampling (FFS) is an established scientific method for sampling rare events in molecular simulations. However, as the difficulty of the scientific problem increases, the amount of data and the number of tasks required for FFS is challenging to manage with traditional scripting tools and languages for high performance computing. The SAFFIRE software framework has been developed to address these challenges. SAFFIRE utilizes Hadoop to manage a large number of tasks and data for large scale FFS simulations. The framework is shown to be highly scalable and able to support large scale FFS simulations. This enables studies of rare events in complex molecular systems on commodity cluster computing systems.

AB - Forward flux sampling (FFS) is an established scientific method for sampling rare events in molecular simulations. However, as the difficulty of the scientific problem increases, the amount of data and the number of tasks required for FFS is challenging to manage with traditional scripting tools and languages for high performance computing. The SAFFIRE software framework has been developed to address these challenges. SAFFIRE utilizes Hadoop to manage a large number of tasks and data for large scale FFS simulations. The framework is shown to be highly scalable and able to support large scale FFS simulations. This enables studies of rare events in complex molecular systems on commodity cluster computing systems.

KW - Data-intensive computing

KW - Forward Flux Sampling

KW - Hadoop

KW - Molecular simulations

KW - Rare events

UR - https://www.scopus.com/pages/publications/85070984400

UR - https://www.scopus.com/pages/publications/85070984400#tab=citedBy

U2 - 10.1145/3332186.3332205

DO - 10.1145/3332186.3332205

M3 - Conference contribution

AN - SCOPUS:85070984400

T3 - ACM International Conference Proceeding Series

BT - Proceedings of the Practice and Experience in Advanced Research Computing

PB - Association for Computing Machinery

T2 - 2019 Conference on Practice and Experience in Advanced Research Computing: Rise of the Machines (Learning), PEARC 2019

Y2 - 28 July 2019 through 1 August 2019

ER -

Building a scalable forward flux sampling framework using big data and H PC

Abstract

Publication series

Conference

Bibliographical note

Keywords

Publisher link

Find It

Other files and links

Fingerprint

Cite this