The role of publicly available data in MICCAI papers from 2014 to 2018

Nicholas Heller, Jack Rickman, Christopher Weight, Nikolaos Papanikolopoulos

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Widely-used public benchmarks are of huge importance to computer vision and machine learning research, especially with the computational resources required to reproduce state of the art results quickly becoming untenable. In medical image computing, the wide variety of image modalities and problem formulations yields a huge task-space for benchmarks to cover, and thus the widespread adoption of standard benchmarks has been slow, and barriers to releasing medical data exacerbate this issue. In this paper, we examine the role that publicly available data has played in MICCAI papers from the past five years. We find that more than half of these papers are based on private data alone, although this proportion seems to be decreasing over time. Additionally, we observed that after controlling for open access publication and the release of code, papers based on public data were cited over 60% more per year than their private-data counterparts. Further, we found that more than 20% of papers using public data did not provide a citation to the dataset or associated manuscript, highlighting the “second-rate” status that data contributions often take compared to theoretical ones. We conclude by making recommendations for MICCAI policies which could help to better incentivise data sharing and move the field toward more efficient and reproducible science.

Original languageEnglish (US)
Title of host publicationLarge-Scale Annotation of Biomedical Data and Expert Label Synthesis and Hardware Aware Learning for Medical Imaging and Computer Assisted Intervention - International Workshops, LABELS 2019, HAL-MICCAI 2019, and CuRIOUS 2019, held in Conjunction with MICCAI 2019, Proceedings
EditorsLuping Zhou, Nicholas Heller, Yiyu Shi, Danny Chen, X. Sharon Hu, Yiming Xiao, Raphael Sznitman, Veronika Cheplygina, Diana Mateus, Emanuele Trucco, Matthieu Chabanas, Hassan Rivaz, Ingerid Reinertsen
PublisherSpringer
Pages70-77
Number of pages8
ISBN (Print)9783030336417
DOIs
StatePublished - Jan 1 2019
Event4th International Workshop on Large-Scale Annotation of Biomedical Data and Expert Label Synthesis, LABELS 2019, the 1st International Workshop on Hardware Aware Learning for Medical Imaging and Computer Assisted Intervention, HAL-MICCAI 2019, and the 2nd International Workshop on Correction of Brainshift with Intra-Operative Ultrasound, CuRIOUS 2019, held in conjunction with the 22nd International Conference on Medical Imaging and Computer-Assisted Intervention, MICCAI 2019 - Shenzhen, China
Duration: Oct 17 2019Oct 17 2019

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11851 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference4th International Workshop on Large-Scale Annotation of Biomedical Data and Expert Label Synthesis, LABELS 2019, the 1st International Workshop on Hardware Aware Learning for Medical Imaging and Computer Assisted Intervention, HAL-MICCAI 2019, and the 2nd International Workshop on Correction of Brainshift with Intra-Operative Ultrasound, CuRIOUS 2019, held in conjunction with the 22nd International Conference on Medical Imaging and Computer-Assisted Intervention, MICCAI 2019
CountryChina
CityShenzhen
Period10/17/1910/17/19

    Fingerprint

Cite this

Heller, N., Rickman, J., Weight, C., & Papanikolopoulos, N. (2019). The role of publicly available data in MICCAI papers from 2014 to 2018. In L. Zhou, N. Heller, Y. Shi, D. Chen, X. S. Hu, Y. Xiao, R. Sznitman, V. Cheplygina, D. Mateus, E. Trucco, M. Chabanas, H. Rivaz, & I. Reinertsen (Eds.), Large-Scale Annotation of Biomedical Data and Expert Label Synthesis and Hardware Aware Learning for Medical Imaging and Computer Assisted Intervention - International Workshops, LABELS 2019, HAL-MICCAI 2019, and CuRIOUS 2019, held in Conjunction with MICCAI 2019, Proceedings (pp. 70-77). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11851 LNCS). Springer. https://doi.org/10.1007/978-3-030-33642-4_8