Imperfect segmentation labels: How much do they matter?

Nicholas Heller, Joshua Dean, Nikolaos Papanikolopoulos

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

Labeled datasets for semantic segmentation are imperfect, especially in medical imaging where borders are often subtle or ill-defined. Little work has been done to analyze the effect that label errors have on the performance of segmentation methodologies. Here we present a large-scale study of model performance in the presence of varying types and degrees of error in training data. We trained U-Net, SegNet, and FCN32 several times for liver segmentation with 10 different modes of ground-truth perturbation. Our results show that for each architecture, performance steadily declines with boundary-localized errors, however, U-Net was significantly more robust to jagged boundary errors than the other architectures. We also found that each architecture was very robust to non-boundary-localized errors, suggesting that boundary-localized errors are fundamentally different and more challenging problem than random label errors in a classification setting.

Original languageEnglish (US)
Title of host publicationIntravascular Imaging and Computer Assisted Stenting and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis - 7th Joint International Workshop, CVII-STENT 2018 and Third International Workshop, LABELS 2018 Held in Conjunction with MICCAI 2018
EditorsSu-Lin Lee, Emanuele Trucco, Lena Maier-Hein, Stefano Moriconi, Shadi Albarqouni, Pierre Jannin, Simone Balocco, Guillaume Zahnd, Diana Mateus, Zeike Taylor, Stefanie Demirci, Danail Stoyanov, Raphael Sznitman, Anne Martel, Veronika Cheplygina, Eric Granger, Luc Duong
PublisherSpringer Verlag
Pages112-120
Number of pages9
ISBN (Print)9783030013639
DOIs
StatePublished - Jan 1 2018
Event7th Joint International Workshop on Computing and Visualization for Intravascular Imaging and Computer Assisted Stenting, CVII-STENT 2018, and the 3rd International Workshop on Large-Scale Annotation of Biomedical Data and Expert Label Synthesis, LABELS 2018, held in conjunction with the 21th International Conference on Medical Imaging and Computer-Assisted Intervention, MICCAI 2018 - Granada, Spain
Duration: Sep 16 2018Sep 16 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11043 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other7th Joint International Workshop on Computing and Visualization for Intravascular Imaging and Computer Assisted Stenting, CVII-STENT 2018, and the 3rd International Workshop on Large-Scale Annotation of Biomedical Data and Expert Label Synthesis, LABELS 2018, held in conjunction with the 21th International Conference on Medical Imaging and Computer-Assisted Intervention, MICCAI 2018
CountrySpain
CityGranada
Period9/16/189/16/18

    Fingerprint

Cite this

Heller, N., Dean, J., & Papanikolopoulos, N. (2018). Imperfect segmentation labels: How much do they matter? In S-L. Lee, E. Trucco, L. Maier-Hein, S. Moriconi, S. Albarqouni, P. Jannin, S. Balocco, G. Zahnd, D. Mateus, Z. Taylor, S. Demirci, D. Stoyanov, R. Sznitman, A. Martel, V. Cheplygina, E. Granger, & L. Duong (Eds.), Intravascular Imaging and Computer Assisted Stenting and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis - 7th Joint International Workshop, CVII-STENT 2018 and Third International Workshop, LABELS 2018 Held in Conjunction with MICCAI 2018 (pp. 112-120). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11043 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-01364-6_13