The fragility of statistically significant findings from randomised controlled trials in the urological literature

Vikram M. Narayan, Shreyas Gandhi, Kristin Chrouser, Nathan Evaniew, Philipp Dahm

Research output: Contribution to journalArticlepeer-review

14 Scopus citations

Abstract

Objectives: To use the Fragility Index to evaluate the robustness of statistically significant findings from urological randomised controlled trials (RCTs). Materials and Methods: The ‘Fragility Index’ is defined as the minimum number of patients in one arm of a trial whose status would have to change from ‘event’ to ‘non-event’, such that a statistically significant result becomes non-significant. We identified all RCTs published in four major urology journals between 2011 and 2015, and we determined the Fragility Index values for those trials reporting statistically significant results of dichotomous outcomes using the Fisher's exact test. Results: In all, 332 RCTs were identified, and 41 studies met the inclusion criteria. The median (interquartile range) Fragility Index was 3 (1, 4.5), indicating that an addition of only three alternate events to one arm of a typical trial would have eliminated its statistical significance. In 27/40 cases (67.5% of cases), the number of patients lost to follow-up was larger than its Fragility Index. Conclusions: The results of urology RCTs that study dichotomous outcomes and report statistically significant differences between groups are sometimes fragile and depend on few events. Urologists should interpret these RCTs cautiously, particularly when the number of participants lost to follow-up exceeds the Fragility Index. Routine reporting of Fragility Index values alongside P values may provide additional guidance about the robustness of statistically significant findings.

Original languageEnglish (US)
Pages (from-to)160-166
Number of pages7
JournalBJU International
Volume122
Issue number1
DOIs
StatePublished - Jul 2018

Bibliographical note

Publisher Copyright:
© 2018 The Authors BJU International © 2018 BJU International Published by John Wiley & Sons Ltd

Copyright:
Copyright 2019 Elsevier B.V., All rights reserved.

Keywords

  • Fragility Index
  • evidence-based medicine
  • randomised controlled trials
  • statistical data interpretation

Fingerprint Dive into the research topics of 'The fragility of statistically significant findings from randomised controlled trials in the urological literature'. Together they form a unique fingerprint.

Cite this