Simulations evaluating resampling methods for causal discovery: Ensemble performance and calibration

Erich Kummerfeld, Alexander Rix

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Causal discovery can be a powerful tool for investigating causality when a system can be observed but is inaccessible to experiments in practice. Despite this, it is rarely used in any scientific or medical fields. One of the major hurdles preventing the field of causal discovery from having a larger impact is that it is difficult to determine when the output of a causal discovery method can be trusted in a real-world setting. Trust is especially critical when human health is on the line. In this paper, we report the results of a series of simulation studies investigating the performance of different resampling methods as indicators of confidence in discovered graph features. We found that subsampling and sampling with replacement both performed surprisingly well, suggesting that they can serve as grounds for confidence in graph features. We also found that the calibration of subsampling and sampling with replacement had different convergence properties, suggesting that one's choice of which to use should depend on the sample size.

Original languageEnglish (US)
Title of host publicationProceedings - 2019 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2019
EditorsIllhoi Yoo, Jinbo Bi, Xiaohua Tony Hu
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages2586-2593
Number of pages8
ISBN (Electronic)9781728118673
DOIs
StatePublished - Nov 2019
Event2019 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2019 - San Diego, United States
Duration: Nov 18 2019Nov 21 2019

Publication series

NameProceedings - 2019 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2019

Conference

Conference2019 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2019
CountryUnited States
CitySan Diego
Period11/18/1911/21/19

Keywords

  • calibration
  • Causal discovery
  • graphical models
  • simulation

Fingerprint Dive into the research topics of 'Simulations evaluating resampling methods for causal discovery: Ensemble performance and calibration'. Together they form a unique fingerprint.

  • Cite this

    Kummerfeld, E., & Rix, A. (2019). Simulations evaluating resampling methods for causal discovery: Ensemble performance and calibration. In I. Yoo, J. Bi, & X. T. Hu (Eds.), Proceedings - 2019 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2019 (pp. 2586-2593). [8983327] (Proceedings - 2019 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BIBM47256.2019.8983327