Bias Analysis Gone Bad

Timothy L. Lash, Thomas P. Ahern, Lindsay J. Collin, Matthew P. Fox, Richard F. MacLehose

Research output: Contribution to journalArticlepeer-review

6 Scopus citations


Quantitative bias analysis comprises the tools used to estimate the direction, magnitude, and uncertainty from systematic errors affecting epidemiologic research. Despite the availability of methods and tools, and guidance for good practices, few reports of epidemiologic research incorporate quantitative estimates of bias impacts. The lack of familiarity with bias analysis allows for the possibility of misuse, which is likely most often unintentional but could occasionally include intentional efforts to mislead. We identified 3 examples of suboptimal bias analysis, one for each common bias. For each, we describe the original research and its bias analysis, compare the bias analysis with good practices, and describe how the bias analysis and research findings might have been improved. We assert no motive to the suboptimal bias analysis by the original authors. Common shortcomings in the examples were lack of a clear bias model, computed example, and computing code; poor selection of the values assigned to the bias model's parameters; and little effort to understand the range of uncertainty associated with the bias. Until bias analysis becomes more common, community expectations for the presentation, explanation, and interpretation of bias analyses will remain unstable. Attention to good practices should improve quality, avoid errors, and discourage manipulation.

Original languageEnglish (US)
Pages (from-to)1604-1612
Number of pages9
JournalAmerican journal of epidemiology
Issue number8
StatePublished - Aug 1 2021

Bibliographical note

Funding Information:
This work was supported by the National Library of Medicine (grant R01LM013049, awarded to T.L.L.). L.J.C. was supported in part by the National Cancer Institute (grant F31CA239566). T.P.A. was supported in part by the National Institute for General Medical Sciences (award P20 GM103644).

Publisher Copyright:
© 2021 The Author(s).


  • epidemiologic bias
  • epidemiologic methods
  • quantitative bias analysis


Dive into the research topics of 'Bias Analysis Gone Bad'. Together they form a unique fingerprint.

Cite this