Observational studies in systemic reviews of comparative effectiveness: AHRQ and the Effective Health Care Program

Susan L. Norris, David Atkins, Wendy Bruening, Steven Fox, Eric Johnson, Robert Kane, Sally C. Morton, Mark Oremus, Maria Ospina, Gurvaneet Randhawa, Karen Schoelles, Paul Shekelle, Meera Viswanathan

Research output: Contribution to journalArticlepeer-review

50 Scopus citations


Objective: Systematic reviewers disagree about the ability of observational studies to answer questions about the benefits or intended effects of pharmacotherapeutic, device, or procedural interventions. This study provides a framework for decision making on the inclusion of observational studies to assess benefits and intended effects in comparative effectiveness reviews (CERs). Study Design and Setting: The conceptual model and recommendations were developed using a consensus process by members of the methods workgroup of the Effective Health Care Program of the Agency for Healthcare Research and Quality. Results: In considering whether to use observational studies in CERs for addressing beneficial effects, reviewers should answer two questions: (1) Are there gaps in the evidence from randomized controlled trials (RCTs)? (2) Will observational studies provide valid and useful information? The latter question involves the following: (a) refocusing the study questions on gaps in the evidence from RCTs, (b) assessing the risk of bias of the body of evidence of observational studies, and (c) assessing whether available observational studies address the gap review questions. Conclusions: Because it is unusual to find sufficient evidence from RCTs to answer all key questions concerning benefit or the balance of benefits and harms, comparative effectiveness reviewers should routinely assess the appropriateness of inclusion of observational studies for questions of benefit. Furthermore, reviewers should explicitly state the rationale for inclusion or exclusion of observational studies when conducting CERs.

Original languageEnglish (US)
Pages (from-to)1178-1186
Number of pages9
JournalJournal of Clinical Epidemiology
Issue number11
StatePublished - Nov 2011


  • Evidence-based medicine, Research design
  • Observational studies
  • Systematic reviews, Meta-analyses, Treatment outcome


Dive into the research topics of 'Observational studies in systemic reviews of comparative effectiveness: AHRQ and the Effective Health Care Program'. Together they form a unique fingerprint.

Cite this