Collaborative experience between scientific software projects using Agile Scrum development

The SCiMMA and SNEWS Collaborations

doi:10.1002/spe.3120

Collaborative experience between scientific software projects using Agile Scrum development

The SCiMMA and SNEWS Collaborations

Physics & Astronomy (Duluth)

Research output: Contribution to journal › Article › peer-review

9 Scopus citations

Abstract

Developing sustainable software for the scientific community requires expertise in software engineering and domain science. This can be challenging due to the unique needs of scientific software, the insufficient resources for software engineering practices in the scientific community, and the complexity of developing for evolving scientific contexts. While open-source software can partially address these concerns, it can introduce complicating dependencies and delay development. These issues can be reduced if scientists and software developers collaborate. We present a case study wherein scientists from the SuperNova Early Warning System collaborated with software developers from the Scalable Cyberinfrastructure for Multi-Messenger Astrophysics project. The collaboration addressed the difficulties of open-source software development, but presented additional risks to each team. For the scientists, there was a concern of relying on external systems and lacking control in the development process. For the developers, there was a risk in supporting a user-group while maintaining core development. These issues were mitigated by creating a second Agile Scrum framework in parallel with the developers' ongoing Agile Scrum process. This Agile collaboration promoted communication, ensured that the scientists had an active role in development, and allowed the developers to evaluate and implement the scientists' software requirements. The collaboration provided benefits for each group: the scientists actuated their development by using an existing platform, and the developers utilized the scientists' use-case to improve their systems. This case study suggests that scientists and software developers can avoid scientific computing issues by collaborating and that Agile Scrum methods can address emergent concerns.

Original language	English (US)
Pages (from-to)	2077-2096
Number of pages	20
Journal	Software - Practice and Experience
Volume	52
Issue number	10
DOIs	https://doi.org/10.1002/spe.3120
State	Published - Oct 2022

Bibliographical note

Funding Information:
The alphabetic‐order author list reflects a variety of contributions from members in the SCiMMA and SNEWS organizations. Particular contributions to this article and the SCiMMA‐SNEWS collaboration are noted as follows. BC, ALB, and RW made primary contributions to the text. BC and ALB led the demonstrations after each phase of development. BC was the Scrum Master who organized and led the development sprints, supported software development during Phases 1 and 2, and implemented the software on cloud infrastructure. ALB served as a SNEWS product owner, supported software development during Phases 0, 1 and 2, and contributed substantially during development sprints. YX led the SNEWS software development process during Phase 0, 1 and 2. PG led the SCiMMA software development during Phases 1 and 2. Together, PG and YX contributed substantially during development sprints. MWL supported software development during Phases 1 and 2, and implemented the software on prototype infrastructure. AH served as a SNEWS product owner and provided substantial input during development sprints. AB, AH, MJ, RFL, CDT and RW each provided substantial input during progress reviews after each phase. AB, MJ, and CDT proposed the initial collaboration at a National Science Foundation “Harnessing the Data Revolution” grant proposal meeting.

Funding Information:
This work is supported by the National Science Foundation “Windows on the Universe: The Era of Multi‐Messenger Astrophysics” program: “WoU‐MMA: Collaborative Research: A Next‐Generation SuperNova Early Warning System for Multimessenger Astronomy” through grants 1914448, 1914409, 1914447, 1914418, 1914410, 1914416, 1914426; and via the “HDR‐Harnessing the Data Revolution” program, grant 1940209; and with the “Office of Advanced Cyberinfrastructure” through grant 1934752.

Funding Information:
information Division of Physics, Grant/Award Numbers: 1914409; 1914410; 1914416; 1914418; 1914426; 1914447; 1914448; Office of Cyberinfrastructure, Grant/Award Numbers: 1934752; 1940209This work is supported by the National Science Foundation “Windows on the Universe: The Era of Multi-Messenger Astrophysics” program: “WoU-MMA: Collaborative Research: A Next-Generation SuperNova Early Warning System for Multimessenger Astronomy” through grants 1914448, 1914409, 1914447, 1914418, 1914410, 1914416, 1914426; and via the “HDR-Harnessing the Data Revolution” program, grant 1940209; and with the “Office of Advanced Cyberinfrastructure” through grant 1934752.

Publisher Copyright:
© 2022 The Authors. Software: Practice and Experience published by John Wiley & Sons Ltd.

Keywords

Agile
cyberinfrastructure
multimessenger astrophysics
scientific computing
software development

Publisher link

10.1002/spe.3120

Find It

Find It at U of M Libraries

Cite this

@article{db8b222a75ce43f2add01650e1438123,

title = "Collaborative experience between scientific software projects using Agile Scrum development",

abstract = "Developing sustainable software for the scientific community requires expertise in software engineering and domain science. This can be challenging due to the unique needs of scientific software, the insufficient resources for software engineering practices in the scientific community, and the complexity of developing for evolving scientific contexts. While open-source software can partially address these concerns, it can introduce complicating dependencies and delay development. These issues can be reduced if scientists and software developers collaborate. We present a case study wherein scientists from the SuperNova Early Warning System collaborated with software developers from the Scalable Cyberinfrastructure for Multi-Messenger Astrophysics project. The collaboration addressed the difficulties of open-source software development, but presented additional risks to each team. For the scientists, there was a concern of relying on external systems and lacking control in the development process. For the developers, there was a risk in supporting a user-group while maintaining core development. These issues were mitigated by creating a second Agile Scrum framework in parallel with the developers' ongoing Agile Scrum process. This Agile collaboration promoted communication, ensured that the scientists had an active role in development, and allowed the developers to evaluate and implement the scientists' software requirements. The collaboration provided benefits for each group: the scientists actuated their development by using an existing platform, and the developers utilized the scientists' use-case to improve their systems. This case study suggests that scientists and software developers can avoid scientific computing issues by collaborating and that Agile Scrum methods can address emergent concerns.",

keywords = "Agile, cyberinfrastructure, multimessenger astrophysics, scientific computing, software development",

author = "{The SCiMMA and SNEWS Collaborations} and Baxter, {Amanda L.} and BenZvi, {Segev Y.} and Walter Bonivento and Adam Brazier and Michael Clark and Alexis Coleiro and David Collom and Marta Colomer-Molla and Bryce Cousins and {Delgado Orellana}, Aliwen and Damien Dornic and Vladislav Ekimtcov and Shereen ElSayed and {Gallo Rosso}, Andrea and Patrick Godwin and Spencer Griswold and Alec Habig and Remington Hill and Shunsaku Horiuchi and Howell, {D. Andrew} and Johnson, {Margaret W.G.} and Mario Juri{\'c} and Kneller, {James P.} and Abigail Kopec and Claudio Kopper and Vladimir Kulikovskiy and Mathieu Lamoureux and Lang, {Rafael F.} and Shengchao Li and Massimiliano Lincetto and Lindy Lindstrom and Linvill, {Mark W.} and Curtis McCully and Jost Migenda and Danny Milisavljevic and Spencer Nelson and Rita Novoseltseva and Erin O'Sullivan and Donald Petravick and Pointon, {Barry W.} and Nirmal Raj and Andrew Renshaw and Janet Rumleskie and Tom Sonley and Ron Tapia and Tseng, {Jeffrey C.L.} and Tunnell, {Christopher D.} and Godefroy Vannoye and Vigorito, {Carlo F.} and Virtue, {Clarence J.}",

note = "Funding Information: The alphabetic‐order author list reflects a variety of contributions from members in the SCiMMA and SNEWS organizations. Particular contributions to this article and the SCiMMA‐SNEWS collaboration are noted as follows. BC, ALB, and RW made primary contributions to the text. BC and ALB led the demonstrations after each phase of development. BC was the Scrum Master who organized and led the development sprints, supported software development during Phases 1 and 2, and implemented the software on cloud infrastructure. ALB served as a SNEWS product owner, supported software development during Phases 0, 1 and 2, and contributed substantially during development sprints. YX led the SNEWS software development process during Phase 0, 1 and 2. PG led the SCiMMA software development during Phases 1 and 2. Together, PG and YX contributed substantially during development sprints. MWL supported software development during Phases 1 and 2, and implemented the software on prototype infrastructure. AH served as a SNEWS product owner and provided substantial input during development sprints. AB, AH, MJ, RFL, CDT and RW each provided substantial input during progress reviews after each phase. AB, MJ, and CDT proposed the initial collaboration at a National Science Foundation “Harnessing the Data Revolution” grant proposal meeting. Funding Information: This work is supported by the National Science Foundation “Windows on the Universe: The Era of Multi‐Messenger Astrophysics” program: “WoU‐MMA: Collaborative Research: A Next‐Generation SuperNova Early Warning System for Multimessenger Astronomy” through grants 1914448, 1914409, 1914447, 1914418, 1914410, 1914416, 1914426; and via the “HDR‐Harnessing the Data Revolution” program, grant 1940209; and with the “Office of Advanced Cyberinfrastructure” through grant 1934752. Funding Information: information Division of Physics, Grant/Award Numbers: 1914409; 1914410; 1914416; 1914418; 1914426; 1914447; 1914448; Office of Cyberinfrastructure, Grant/Award Numbers: 1934752; 1940209This work is supported by the National Science Foundation “Windows on the Universe: The Era of Multi-Messenger Astrophysics” program: “WoU-MMA: Collaborative Research: A Next-Generation SuperNova Early Warning System for Multimessenger Astronomy” through grants 1914448, 1914409, 1914447, 1914418, 1914410, 1914416, 1914426; and via the “HDR-Harnessing the Data Revolution” program, grant 1940209; and with the “Office of Advanced Cyberinfrastructure” through grant 1934752. Publisher Copyright: {\textcopyright} 2022 The Authors. Software: Practice and Experience published by John Wiley & Sons Ltd.",

year = "2022",

month = oct,

doi = "10.1002/spe.3120",

language = "English (US)",

volume = "52",

pages = "2077--2096",

journal = "Software - Practice and Experience",

issn = "0038-0644",

publisher = "John Wiley and Sons Ltd",

number = "10",

}

TY - JOUR

T1 - Collaborative experience between scientific software projects using Agile Scrum development

AU - The SCiMMA and SNEWS Collaborations

AU - Baxter, Amanda L.

AU - BenZvi, Segev Y.

AU - Bonivento, Walter

AU - Brazier, Adam

AU - Clark, Michael

AU - Coleiro, Alexis

AU - Collom, David

AU - Colomer-Molla, Marta

AU - Cousins, Bryce

AU - Delgado Orellana, Aliwen

AU - Dornic, Damien

AU - Ekimtcov, Vladislav

AU - ElSayed, Shereen

AU - Gallo Rosso, Andrea

AU - Godwin, Patrick

AU - Griswold, Spencer

AU - Habig, Alec

AU - Hill, Remington

AU - Horiuchi, Shunsaku

AU - Howell, D. Andrew

AU - Johnson, Margaret W.G.

AU - Jurić, Mario

AU - Kneller, James P.

AU - Kopec, Abigail

AU - Kopper, Claudio

AU - Kulikovskiy, Vladimir

AU - Lamoureux, Mathieu

AU - Lang, Rafael F.

AU - Li, Shengchao

AU - Lincetto, Massimiliano

AU - Lindstrom, Lindy

AU - Linvill, Mark W.

AU - McCully, Curtis

AU - Migenda, Jost

AU - Milisavljevic, Danny

AU - Nelson, Spencer

AU - Novoseltseva, Rita

AU - O'Sullivan, Erin

AU - Petravick, Donald

AU - Pointon, Barry W.

AU - Raj, Nirmal

AU - Renshaw, Andrew

AU - Rumleskie, Janet

AU - Sonley, Tom

AU - Tapia, Ron

AU - Tseng, Jeffrey C.L.

AU - Tunnell, Christopher D.

AU - Vannoye, Godefroy

AU - Vigorito, Carlo F.

AU - Virtue, Clarence J.

N1 - Funding Information: The alphabetic‐order author list reflects a variety of contributions from members in the SCiMMA and SNEWS organizations. Particular contributions to this article and the SCiMMA‐SNEWS collaboration are noted as follows. BC, ALB, and RW made primary contributions to the text. BC and ALB led the demonstrations after each phase of development. BC was the Scrum Master who organized and led the development sprints, supported software development during Phases 1 and 2, and implemented the software on cloud infrastructure. ALB served as a SNEWS product owner, supported software development during Phases 0, 1 and 2, and contributed substantially during development sprints. YX led the SNEWS software development process during Phase 0, 1 and 2. PG led the SCiMMA software development during Phases 1 and 2. Together, PG and YX contributed substantially during development sprints. MWL supported software development during Phases 1 and 2, and implemented the software on prototype infrastructure. AH served as a SNEWS product owner and provided substantial input during development sprints. AB, AH, MJ, RFL, CDT and RW each provided substantial input during progress reviews after each phase. AB, MJ, and CDT proposed the initial collaboration at a National Science Foundation “Harnessing the Data Revolution” grant proposal meeting. Funding Information: This work is supported by the National Science Foundation “Windows on the Universe: The Era of Multi‐Messenger Astrophysics” program: “WoU‐MMA: Collaborative Research: A Next‐Generation SuperNova Early Warning System for Multimessenger Astronomy” through grants 1914448, 1914409, 1914447, 1914418, 1914410, 1914416, 1914426; and via the “HDR‐Harnessing the Data Revolution” program, grant 1940209; and with the “Office of Advanced Cyberinfrastructure” through grant 1934752. Funding Information: information Division of Physics, Grant/Award Numbers: 1914409; 1914410; 1914416; 1914418; 1914426; 1914447; 1914448; Office of Cyberinfrastructure, Grant/Award Numbers: 1934752; 1940209This work is supported by the National Science Foundation “Windows on the Universe: The Era of Multi-Messenger Astrophysics” program: “WoU-MMA: Collaborative Research: A Next-Generation SuperNova Early Warning System for Multimessenger Astronomy” through grants 1914448, 1914409, 1914447, 1914418, 1914410, 1914416, 1914426; and via the “HDR-Harnessing the Data Revolution” program, grant 1940209; and with the “Office of Advanced Cyberinfrastructure” through grant 1934752. Publisher Copyright: © 2022 The Authors. Software: Practice and Experience published by John Wiley & Sons Ltd.

PY - 2022/10

Y1 - 2022/10

N2 - Developing sustainable software for the scientific community requires expertise in software engineering and domain science. This can be challenging due to the unique needs of scientific software, the insufficient resources for software engineering practices in the scientific community, and the complexity of developing for evolving scientific contexts. While open-source software can partially address these concerns, it can introduce complicating dependencies and delay development. These issues can be reduced if scientists and software developers collaborate. We present a case study wherein scientists from the SuperNova Early Warning System collaborated with software developers from the Scalable Cyberinfrastructure for Multi-Messenger Astrophysics project. The collaboration addressed the difficulties of open-source software development, but presented additional risks to each team. For the scientists, there was a concern of relying on external systems and lacking control in the development process. For the developers, there was a risk in supporting a user-group while maintaining core development. These issues were mitigated by creating a second Agile Scrum framework in parallel with the developers' ongoing Agile Scrum process. This Agile collaboration promoted communication, ensured that the scientists had an active role in development, and allowed the developers to evaluate and implement the scientists' software requirements. The collaboration provided benefits for each group: the scientists actuated their development by using an existing platform, and the developers utilized the scientists' use-case to improve their systems. This case study suggests that scientists and software developers can avoid scientific computing issues by collaborating and that Agile Scrum methods can address emergent concerns.

AB - Developing sustainable software for the scientific community requires expertise in software engineering and domain science. This can be challenging due to the unique needs of scientific software, the insufficient resources for software engineering practices in the scientific community, and the complexity of developing for evolving scientific contexts. While open-source software can partially address these concerns, it can introduce complicating dependencies and delay development. These issues can be reduced if scientists and software developers collaborate. We present a case study wherein scientists from the SuperNova Early Warning System collaborated with software developers from the Scalable Cyberinfrastructure for Multi-Messenger Astrophysics project. The collaboration addressed the difficulties of open-source software development, but presented additional risks to each team. For the scientists, there was a concern of relying on external systems and lacking control in the development process. For the developers, there was a risk in supporting a user-group while maintaining core development. These issues were mitigated by creating a second Agile Scrum framework in parallel with the developers' ongoing Agile Scrum process. This Agile collaboration promoted communication, ensured that the scientists had an active role in development, and allowed the developers to evaluate and implement the scientists' software requirements. The collaboration provided benefits for each group: the scientists actuated their development by using an existing platform, and the developers utilized the scientists' use-case to improve their systems. This case study suggests that scientists and software developers can avoid scientific computing issues by collaborating and that Agile Scrum methods can address emergent concerns.

KW - Agile

KW - cyberinfrastructure

KW - multimessenger astrophysics

KW - scientific computing

KW - software development

UR - http://www.scopus.com/inward/record.url?scp=85135158068&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85135158068&partnerID=8YFLogxK

U2 - 10.1002/spe.3120

DO - 10.1002/spe.3120

M3 - Article

AN - SCOPUS:85135158068

SN - 0038-0644

VL - 52

SP - 2077

EP - 2096

JO - Software - Practice and Experience

JF - Software - Practice and Experience

IS - 10

ER -

Collaborative experience between scientific software projects using Agile Scrum development

Abstract

Bibliographical note

Keywords

Publisher link

Find It

Other files and links

Fingerprint

Cite this