Effect Estimates in Randomized Trials and Observational Studies: Comparing Apples with Apples

Sara Lodi, Andrew Phillips, Jens Lundgren, Roger Logan, Shweta Sharma, Stephen R. Cole, Abdel Babiker, Matthew Law, Haitao Chu, Dana Byrne, Andrzej Horban, Jonathan A.C. Sterne, Kholoud Porter, Caroline Sabin, Dominique Costagliola, Sophie Abgrall, John Gill, Giota Touloumi, Antonio G. Pacheco, Ard Van SighemPeter Reiss, Heiner C. Bucher, Alexandra Montoliu Giménez, Inmaculada Jarrin, Linda Wittkop, Laurence Meyer, Santiago Perez-Hoyos, Amy Justice, James D. Neaton, Miguel A. Hernán

Research output: Contribution to journalArticlepeer-review

61 Scopus citations


Effect estimates from randomized trials and observational studies might not be directly comparable because of differences in study design, other than randomization, and in data analysis. We propose a 3-step procedure to facilitate meaningful comparisons of effect estimates from randomized trials and observational studies: 1) harmonization of the study protocols (eligibility criteria, treatment strategies, outcome, start and end of follow-up, causal contrast) so that the studies target the same causal effect, 2) harmonization of the data analysis to estimate the causal effect, and 3) sensitivity analyses to investigate the impact of discrepancies that could not be accounted for in the harmonization process. To illustrate our approach, we compared estimates of the effect of immediate with deferred initiation of antiretroviral therapy in individuals positive for the human immunodeficiency virus from the Strategic Timing of Antiretroviral Therapy (START) randomized trial and the observational HIV-CAUSAL Collaboration.

Original languageEnglish (US)
Pages (from-to)1569-1577
Number of pages9
JournalAmerican journal of epidemiology
Issue number8
StatePublished - Aug 1 2019

Bibliographical note

Funding Information:
Sport, during the conduct of the study, and grants from European Centre for Disease Prevention and Control, outside the submitted work. G.T. has received grants unrelated to this study from Gilead Sciences Europe, UCL, ECDC, and EU and National funds; D.C. reports grants from Janssen-Cilag, Merck-Sharp & Dohme-Chibret, and ViiV, as well as personal fees from Janssen-Cilagand Merck-Sharp & Dohme-Chibret for lectures, personal fees from ViiV for travel/accommodations/meeting expenses, personal fees from Gilead France for French HIV board, and personal fees from Innavirvax and Merck Switzerland for consultancy, outside the submitted work. P.R. has received independent scientific grant support through his institution from Gilead Sciences, Janssen Pharmaceuticals Inc., Merck & Co., and ViiV Healthcare; he has served on scientific advisory boards for Gilead Sciences, ViiV Healthcare, Merck & Co., Teva Pharmaceutical Industries, and on a data-safety monitoring committee for Janssen Pharmaceuticals Inc., for which his institution has received remuneration. The other authors report no conflicts.

Funding Information:
Author affiliations: Department of Biostatistics, Boston University School of Public Health, Boston, Massachusetts (Sara Lodi); Institute for Global Health, University College London, United Kingdom (Andrew Phillips, Kholoud Porter, Caroline Sabin); Department of Infectious Diseases, Rigshospitalet, University of Copenhagen, Denmark (Jens Lundgren); Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, Massachusetts (Roger Logan, Miguel A. Hernán); Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota (Shweta Sharma, Haitao Chu, James D. Neaton); Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina (Stephen R. Cole); Medical Research Council Clinical Trials Unit in University College London, London, United Kingdom (Abdel Babiker); The Kirby Institute, Sidney, Australia (Matthew Law); Division of Infectious Diseases, Department of Medicine, Cooper University Hospital, Cooper Medical School at Rowan University, Camden, New Jersey (Dana Byrne); Department for Adult’s Infectious Diseases, Medical University of Warsaw, Warsaw, Poland (Andrzej Horban); Department of Population Health Sciences, University of Bristol, Bristol, United Kingdom (Jonathan A. C. Sterne); Institut Pierre Louis d’Épidémiologie et de Santé Publique, Institut National de la Santé et de la Recherche Médicale, Sorbonne Université, Paris, France (Dominique Costagliola, Sophie Abgrall); Service de Médecine Interne, Hôpital Antoine Béclère, AP-HP, Clamart, France (Sophie Abgrall); Southern Alberta Clinic, Calgary, Canada (John Gill); Department of Medicine, Faculty of Medicine, University of Calgary, Canada (John Gill); Department of Hygiene, Epidemiology and Medical Statistics, Faculty of Medicine, National and Kapodistrian University of Athens, Greece (Giota Touloumi); Programa de Computação Científica, Fundacao Oswaldo Cruz, Rio de Janeiro, Brazil (Antonio G. Pacheco); Stichting HIV Monitoring, Amsterdam, the Netherlands (Ard van Sighem, Peter Reiss); Department of Global Health, Amsterdam University Medical Center, University of Amsterdam, Amsterdam, the Netherlands (Peter Reiss); Division of Infectious Diseases, University of Amsterdam, Amsterdam, the Netherlands (Peter Reiss); Amsterdam Institute for Global Health and Development, Amsterdam, the Netherlands (Peter Reiss); Amsterdam Public Health Research Institute, Amsterdam, the Netherlands (Peter Reiss); Basel Institute for Clinical Epidemiology and Biostatistics, University Hospital Basel, University of Basel, Switzerland (Heiner C. Bucher); Centre for Epidemiological Studies on HIV/STI in Catalonia, Agència de Salut Pública de Catalunya, Badalona, Spain (Alexandra Montoliu Giménez); Centro Nacional de Epidemiología, Instituto de Salud Carlos III, Madrid, Spain (Inmaculada Jarrin); Bordeaux Population Health Research Center, Team MORPH3EUS, Unité Mixte de Recherche 1219, CIC-EC 1401, Institut de Santé Publique, d’Epidémiologie et de Développement, Institut National de la Santé et de la Recherche Médicale, University of Bordeaux, Bordeaux, France (Linda Wittkop); Service D’information Médicale, Pôle de Santé Publique, Centre Hospitalier Universitaire de Bordeaux, Bordeaux, France (Laurence Meyer); Unité Mixte de Recherche 1018, le Kremlin Bicêtre, Université Paris Sud, Paris, France (Laurence Meyer); Vall d’Hebrón Research Institute, Barcelona, Spain (Santiago Perez-Hoyos); Department of Internal Medicine, Yale University School of Medicine, New Haven, Connecticut (Amy Justice); Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, Massachusetts (Miguel A. Hernán); and Harvard-MIT Division of Health Sciences and Technology, Boston, Massachusetts (Miguel A. Hernán). This work received funding from Harvard University Center for AIDS Research (grant 5P30AI060354-13) and the National Institutes of Health (grants AI102634, UL1TR001079, UM1-AI068641 and UM1-AI120197). The funding organizations had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript. The views expressed are those of the author(s) and do not necessarily reflect the official views of the Uniformed Services University of the Health Sciences, the National Institutes of Health, the Department of Defense, or the Departments of the Army, Navy or Air Force. A complete list of contributors to the HIV-CAUSAL Collaboration and of the INSIGHT START group can be found in Web Appendices 1 and 2.

Funding Information:
A.R. has received funding from the Bill & Melinda Gates Foundation; H.R. and his institution have received honoraria, support to attend conferences, or unrestricted research grants from Gilead Sciences, BMS, ViiV Healthcare, Janssen, Abbvie, and MSD in the 3 years preceding the submission date of this manuscript; C.S. received funding from Gilead Sciences, ViiV Healthcare, and Janssen-Cilag for membership in Data Safety and Monitoring Boards, Advisory Boards, and Speaker Panels and for the preparation of educational materials; A.v.S. reports grants from Dutch Ministry of Health, Welfare and

Publisher Copyright:
© 2019 The Author(s). Published by Oxford University Press on behalf of the Johns Hopkins Bloomberg School of Public Health. All rights reserved.


  • antiretroviral initiation
  • causal inference
  • per-protocol effect
  • target trial


Dive into the research topics of 'Effect Estimates in Randomized Trials and Observational Studies: Comparing Apples with Apples'. Together they form a unique fingerprint.

Cite this