Challenges and recommendations to improve the installability and archival stability of omics computational tools

Serghei Mangul; Thiago Mosqueiro; Richard J. Abdill; Dat Duong; Keith Mitchell; Varuni Sarwal; Brian Hill; Jaqueline Brito; Russell Jared Littman; Benjamin Statz; Angela Ka Mei Lam; Gargi Dayama; Laura Grieneisen; Lana S. Martin; Jonathan Flint; Eleazar Eskin; Ran Blekhman

doi:10.1371/journal.pbio.3000333

Challenges and recommendations to improve the installability and archival stability of omics computational tools

Serghei Mangul, Thiago Mosqueiro, Richard J. Abdill, Dat Duong, Keith Mitchell, Varuni Sarwal, Brian Hill, Jaqueline Brito, Russell Jared Littman, Benjamin Statz, Angela Ka Mei Lam, Gargi Dayama, Laura Grieneisen, Lana S. Martin, Jonathan Flint, Eleazar Eskin, Ran Blekhman

Research output: Contribution to journal › Article › peer-review

43 Scopus citations

Abstract

Developing new software tools for analysis of large-scale biological data is a key component of advancing modern biomedical research. Scientific reproduction of published findings requires running computational tools on data generated by such studies, yet little attention is presently allocated to the installability and archival stability of computational software tools. Scientific journals require data and code sharing, but none currently require authors to guarantee the continuing functionality of newly published tools. We have estimated the archival stability of computational biology software tools by performing an empirical analysis of the internet presence for 36,702 omics software resources published from 2005 to 2017. We found that almost 28% of all resources are currently not accessible through uniform resource locators (URLs) published in the paper they first appeared in. Among the 98 software tools selected for our installability test, 51% were deemed “easy to install,” and 28% of the tools failed to be installed at all because of problems in the implementation. Moreover, for papers introducing new software, we found that the number of citations significantly increased when authors provided an easy installation process. We propose for incorporation into journal policy several practical solutions for increasing the widespread installability and archival stability of published bioinformatics software.

Original language	English (US)
Article number	e3000333
Journal	PLoS biology
Volume	17
Issue number	6
DOIs	https://doi.org/10.1371/journal.pbio.3000333
State	Published - Jun 2019

Bibliographical note

Funding Information:
SM acknowledges support from a QCB Collaboratory Postdoctoral Fellowship and the QCB Collaboratory community directed by Matteo Pellegrini. SM and EE are supported by National Science Foundation grants 0513612, 0731455, 0729049, 0916676, 1065276, 1302448, 1320589, 1331176, and 1815624 and National Institutes of Health grants K25-HL080079, U01-DA024417, P01-HL30568, P01-HL28481, R01-GM083198, R01-ES021801, R01-MH101782, and R01-ES022282. RB is grateful for support from the National Institutes of General Medicine (R35-GM128716) and a McKnight Land-Grant Professorship from the University of Minnesota.

Publisher link

10.1371/journal.pbio.3000333

Find It

Find It at U of M Libraries

Cite this

Mangul, S., Mosqueiro, T., Abdill, R. J., Duong, D., Mitchell, K., Sarwal, V., Hill, B., Brito, J., Littman, R. J., Statz, B., Lam, A. K. M., Dayama, G., Grieneisen, L., Martin, L. S., Flint, J., Eskin, E., & Blekhman, R. (2019). Challenges and recommendations to improve the installability and archival stability of omics computational tools. PLoS biology, 17(6), Article e3000333. https://doi.org/10.1371/journal.pbio.3000333

Mangul, S, Mosqueiro, T, Abdill, RJ, Duong, D, Mitchell, K, Sarwal, V, Hill, B, Brito, J, Littman, RJ, Statz, B, Lam, AKM, Dayama, G, Grieneisen, L, Martin, LS, Flint, J, Eskin, E & Blekhman, R 2019, 'Challenges and recommendations to improve the installability and archival stability of omics computational tools', PLoS biology, vol. 17, no. 6, e3000333. https://doi.org/10.1371/journal.pbio.3000333

@article{5af70c27b7c14dab983333a1485c377f,

title = "Challenges and recommendations to improve the installability and archival stability of omics computational tools",

abstract = "Developing new software tools for analysis of large-scale biological data is a key component of advancing modern biomedical research. Scientific reproduction of published findings requires running computational tools on data generated by such studies, yet little attention is presently allocated to the installability and archival stability of computational software tools. Scientific journals require data and code sharing, but none currently require authors to guarantee the continuing functionality of newly published tools. We have estimated the archival stability of computational biology software tools by performing an empirical analysis of the internet presence for 36,702 omics software resources published from 2005 to 2017. We found that almost 28% of all resources are currently not accessible through uniform resource locators (URLs) published in the paper they first appeared in. Among the 98 software tools selected for our installability test, 51% were deemed “easy to install,” and 28% of the tools failed to be installed at all because of problems in the implementation. Moreover, for papers introducing new software, we found that the number of citations significantly increased when authors provided an easy installation process. We propose for incorporation into journal policy several practical solutions for increasing the widespread installability and archival stability of published bioinformatics software.",

author = "Serghei Mangul and Thiago Mosqueiro and Abdill, {Richard J.} and Dat Duong and Keith Mitchell and Varuni Sarwal and Brian Hill and Jaqueline Brito and Littman, {Russell Jared} and Benjamin Statz and Lam, {Angela Ka Mei} and Gargi Dayama and Laura Grieneisen and Martin, {Lana S.} and Jonathan Flint and Eleazar Eskin and Ran Blekhman",

note = "Funding Information: SM acknowledges support from a QCB Collaboratory Postdoctoral Fellowship and the QCB Collaboratory community directed by Matteo Pellegrini. SM and EE are supported by National Science Foundation grants 0513612, 0731455, 0729049, 0916676, 1065276, 1302448, 1320589, 1331176, and 1815624 and National Institutes of Health grants K25-HL080079, U01-DA024417, P01-HL30568, P01-HL28481, R01-GM083198, R01-ES021801, R01-MH101782, and R01-ES022282. RB is grateful for support from the National Institutes of General Medicine (R35-GM128716) and a McKnight Land-Grant Professorship from the University of Minnesota.",

year = "2019",

month = jun,

doi = "10.1371/journal.pbio.3000333",

language = "English (US)",

volume = "17",

journal = "PLoS biology",

issn = "1544-9173",

publisher = "Public Library of Science",

number = "6",

}

TY - JOUR

T1 - Challenges and recommendations to improve the installability and archival stability of omics computational tools

AU - Mangul, Serghei

AU - Mosqueiro, Thiago

AU - Abdill, Richard J.

AU - Duong, Dat

AU - Mitchell, Keith

AU - Sarwal, Varuni

AU - Hill, Brian

AU - Brito, Jaqueline

AU - Littman, Russell Jared

AU - Statz, Benjamin

AU - Lam, Angela Ka Mei

AU - Dayama, Gargi

AU - Grieneisen, Laura

AU - Martin, Lana S.

AU - Flint, Jonathan

AU - Eskin, Eleazar

AU - Blekhman, Ran

N1 - Funding Information: SM acknowledges support from a QCB Collaboratory Postdoctoral Fellowship and the QCB Collaboratory community directed by Matteo Pellegrini. SM and EE are supported by National Science Foundation grants 0513612, 0731455, 0729049, 0916676, 1065276, 1302448, 1320589, 1331176, and 1815624 and National Institutes of Health grants K25-HL080079, U01-DA024417, P01-HL30568, P01-HL28481, R01-GM083198, R01-ES021801, R01-MH101782, and R01-ES022282. RB is grateful for support from the National Institutes of General Medicine (R35-GM128716) and a McKnight Land-Grant Professorship from the University of Minnesota.

PY - 2019/6

Y1 - 2019/6

N2 - Developing new software tools for analysis of large-scale biological data is a key component of advancing modern biomedical research. Scientific reproduction of published findings requires running computational tools on data generated by such studies, yet little attention is presently allocated to the installability and archival stability of computational software tools. Scientific journals require data and code sharing, but none currently require authors to guarantee the continuing functionality of newly published tools. We have estimated the archival stability of computational biology software tools by performing an empirical analysis of the internet presence for 36,702 omics software resources published from 2005 to 2017. We found that almost 28% of all resources are currently not accessible through uniform resource locators (URLs) published in the paper they first appeared in. Among the 98 software tools selected for our installability test, 51% were deemed “easy to install,” and 28% of the tools failed to be installed at all because of problems in the implementation. Moreover, for papers introducing new software, we found that the number of citations significantly increased when authors provided an easy installation process. We propose for incorporation into journal policy several practical solutions for increasing the widespread installability and archival stability of published bioinformatics software.

AB - Developing new software tools for analysis of large-scale biological data is a key component of advancing modern biomedical research. Scientific reproduction of published findings requires running computational tools on data generated by such studies, yet little attention is presently allocated to the installability and archival stability of computational software tools. Scientific journals require data and code sharing, but none currently require authors to guarantee the continuing functionality of newly published tools. We have estimated the archival stability of computational biology software tools by performing an empirical analysis of the internet presence for 36,702 omics software resources published from 2005 to 2017. We found that almost 28% of all resources are currently not accessible through uniform resource locators (URLs) published in the paper they first appeared in. Among the 98 software tools selected for our installability test, 51% were deemed “easy to install,” and 28% of the tools failed to be installed at all because of problems in the implementation. Moreover, for papers introducing new software, we found that the number of citations significantly increased when authors provided an easy installation process. We propose for incorporation into journal policy several practical solutions for increasing the widespread installability and archival stability of published bioinformatics software.

UR - http://www.scopus.com/inward/record.url?scp=85069265675&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85069265675&partnerID=8YFLogxK

U2 - 10.1371/journal.pbio.3000333

DO - 10.1371/journal.pbio.3000333

M3 - Article

C2 - 31220077

AN - SCOPUS:85069265675

SN - 1544-9173

VL - 17

JO - PLoS biology

JF - PLoS biology

IS - 6

M1 - e3000333

ER -

Challenges and recommendations to improve the installability and archival stability of omics computational tools

Abstract

Bibliographical note

Publisher link

Find It

Other files and links

Fingerprint

Cite this