Bias Analysis Gone Bad

Timothy L. Lash, Thomas P. Ahern, Lindsay J. Collin, Matthew P. Fox, Richard F. MacLehose

Research output: Contribution to journalArticlepeer-review


Quantitative bias analysis comprises the tools used to estimate the direction, magnitude, and uncertainty from systematic errors affecting epidemiologic research. Despite the availability of methods and tools, and guidance for good practices, few reports of epidemiologic research incorporate quantitative estimates of bias impacts. The lack of familiarity with bias analysis allows for the possibility of misuse, which is likely most often unintentional but could occasionally include intentional efforts to mislead. We identified 3 examples of suboptimal bias analysis, one for each common bias. For each, we describe the original research and its bias analysis, compare the bias analysis with good practices, and describe how the bias analysis and research findings might have been improved. We assert no motive to the suboptimal bias analysis by the original authors. Common shortcomings in the examples were lack of a clear bias model, computed example, and computing code; poor selection of the values assigned to the bias model's parameters; and little effort to understand the range of uncertainty associated with the bias. Until bias analysis becomes more common, community expectations for the presentation, explanation, and interpretation of bias analyses will remain unstable. Attention to good practices should improve quality, avoid errors, and discourage manipulation.

Original languageEnglish (US)
Pages (from-to)1604-1612
Number of pages9
JournalAmerican journal of epidemiology
Issue number8
StatePublished - Aug 1 2021

Bibliographical note

Publisher Copyright:
© The Author(s) 2021. Published by Oxford University Press on behalf of the Johns Hopkins Bloomberg School of Public Health. All rights reserved. For permissions, please e-mail:


  • epidemiologic bias
  • epidemiologic methods
  • quantitative bias analysis


Dive into the research topics of 'Bias Analysis Gone Bad'. Together they form a unique fingerprint.

Cite this