Spatialising uncertainty in image segmentation using weakly supervised convolutional neural networks: A case study from historical map processing

Johannes H. Uhl, Stefan Leyk, Yao Yi Chiang, Weiwei Duan, Craig A. Knoblock

Research output: Contribution to journalArticlepeer-review

14 Scopus citations


Convolutional neural networks (CNNs) such as encoder-decoder CNNs have increasingly been employed for semantic image segmentation at the pixel-level requiring pixel-level training labels, which are rarely available in real-world scenarios. In practice, weakly annotated training data at the image patch level are often used for pixel-level segmentation tasks, requiring further processing to obtain accurate results, mainly because the translation invariance of the CNN-based inference can turn into an impeding property leading to segmentation results of coarser spatial granularity compared with the original image. However, the inherent uncertainty in the segmented image and its relationships to translation invariance, CNN architecture, and classification scheme has never been analysed from an explicitly spatial perspective. Therefore, the authors propose measures to spatially visualise and assess class decision confidence based on spatially dense CNN predictions, resulting in continuous decision confidence surfaces. They find that such a visual-analytical method contributes to a better understanding of the spatial variability of class score confidence derived from weakly supervised CNN-based classifiers. They exemplify this approach by incorporating decision confidence surfaces into a processing chain for the extraction of human settlement features from historical map documents based on weakly annotated training data using different CNN architectures and classification schemes.

Original languageEnglish (US)
Pages (from-to)2084-2091
Number of pages8
JournalIET Image Processing
Issue number11
StatePublished - Nov 1 2018
Externally publishedYes

Bibliographical note

Funding Information:
This material is based on research sponsored in part by the National Science Foundation under grant nos. IIS 1563933 (to the University of Colorado at Boulder) and IIS 1564164 (to the University of Southern California). The authors were provided access to the Zillow Transaction and Assessment Dataset (ZTRAX) through a data use agreement between the University of Colorado Boulder and Zillow Inc. Support by Zillow Inc. is gratefully acknowledged.

Publisher Copyright:
© The Institution of Engineering and Technology 2018.


Dive into the research topics of 'Spatialising uncertainty in image segmentation using weakly supervised convolutional neural networks: A case study from historical map processing'. Together they form a unique fingerprint.

Cite this