Innovative Method to Build Robust Prediction Models When Gold-Standard Outcomes Are Scarce

Ying Zhu, Roshan Tourani, Adam Sheka, Elizabeth Wick, Genevieve B. Melton, Gyorgy Simon

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations


Postsurgical hospital acquired infections (HAIs) are viewed as a quality benchmark in healthcare due to their association with morbidity, mortality and high cost. The prediction of HAIs allows for implementing prevention strategies at early stages to reduce postsurgical complications. In the United States, the National Surgical Quality Improvement Program (NSQIP) maintains a registry considered a “gold-standard” for HAIs outcome reporting, but it relies heavily on costly manual chart review and therefore only includes a small percent of surgery cases from each participating site. Most HAI prediction models rely on a wide range of weak risk factors, which are combined into models with many parameters and require larger sample sizes than available from NSQIP at a single health system. In this study, we propose an alternative approach to develop a robust prediction model, using the few NSQIP cases efficiently. Rather than training the HAIs prediction models directly on the small number of NSQIP patients, we leverage a simple detection model which detects HAIs after the fact on postoperative data and use this detection model to label a large non-NSQIP perioperative dataset on which prediction models are constructed. Detection models rely on strong signals requiring fewer samples to learn. We evaluate this approach in a single academic health system with 115,202 surgeries (10,354 in NSQIP). The prediction models were evaluated on the NSQIP “gold-standard” labels. While organ-space surgical site infection showed comparable performance, the proposed model demonstrated better performance for prediction of superficial surgical site infection, sepsis or septic shock, pneumonia, and urinary tract infection.

Original languageEnglish (US)
Title of host publicationArtificial Intelligence in Medicine - 18th International Conference on Artificial Intelligence in Medicine, AIME 2020, Proceedings
EditorsMartin Michalowski, Robert Moskovitch
PublisherSpringer Science and Business Media Deutschland GmbH
Number of pages11
ISBN (Print)9783030591366
StatePublished - 2020
Event18th International Conference on Artificial Intelligence in Medicine, AIME 2020 - Minneapolis, United States
Duration: Aug 25 2020Aug 28 2020

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12299 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference18th International Conference on Artificial Intelligence in Medicine, AIME 2020
Country/TerritoryUnited States

Bibliographical note

Funding Information:
Acknowledgements. This work was supported in part by NIGMS award R01 GM 120079, AHRQ award R01 HS024532, and the NCATS University of Minnesota CTSA UL1 TR002494. The views expressed in this manuscript are those of the authors and do not necessarily reflect the views of the funding agencies.

Publisher Copyright:
© 2020, Springer Nature Switzerland AG.


  • Artificial intelligence
  • Hospital acquired infection
  • Machine learning
  • Medicine
  • Predictive modeling


Dive into the research topics of 'Innovative Method to Build Robust Prediction Models When Gold-Standard Outcomes Are Scarce'. Together they form a unique fingerprint.

Cite this