Random effects models in a meta-analysis of the accuracy of two diagnostic tests without a gold standard

Haitao Chu, Sining Chen, Thomas A. Louis

Research output: Contribution to journalArticlepeer-review

52 Scopus citations


In studies of the accuracy of diagnostic tests, it is common that both the diagnostic test itself and the reference test are imperfect. This is the case for the microsatellite instability test, which is routinely used as a prescreening procedure to identify individuals with Lynch syndrome, the most common hereditary colorectal cancer syndrome. The microsatellite instability test is known to have imperfect sensitivity and specificity. Meanwhile, the reference test, mutation analysis, is also imperfect.We evaluate this test via a random effects meta-analysis of 17 studies. Study-specific random effects account for between-study heterogeneity in mutation prevalence, test sensitivities and specificities under a nonlinear mixed effects model and a Bayesian hierarchical model. Using model selection techniques, we explore a range of random effects models to identify a best-fitting model. We also evaluate sensitivity to the conditional independence assumption between the microsatellite instability test and the mutation analysis by allowing for correlation between them. Finally, we use simulations to illustrate the importance of including appropriate random effects and the impact of overfitting, underfitting, and misfitting on model performance. Our approach can be used to estimate the accuracy of two imperfect diagnostic tests from a meta-analysis of multiple studies or a multicenter study when the prevalence of disease, test sensitivities and/or specificities may be heterogeneous among studies or centers.

Original languageEnglish (US)
Pages (from-to)512-523
Number of pages12
JournalJournal of the American Statistical Association
Issue number486
StatePublished - Jun 2009

Bibliographical note

Funding Information:
Haitao Chu is Research Associate Professor, Department of Biostatistics and Lineberger Comprehensive Cancer Center, The Univerity of North Carolina, Chapel Hill, NC 27599 (E-mail: hchu@bios.unc.edu). Sining Chen is Assistant Professor, Department of Environment Health Sciences, The Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21205 (E-mail: sichen@ jhsph.edu). Thomas A. Louis is Professor, Department of Biostatistics, The Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21205 (E-mail: tlouis@jhsph.edu). Support for T. A. Louis was provided by R01 DK061662 from the U.S. National Institute of Diabetes, Digestive and Kidney Diseases. Haitao Chu is supported in part by the Lineberger Cancer Center Core Grant CA16086 and P50 CA106991 from the U.S. National Cancer Institute. The authors thank the editor, the associate editor, referees, and Dr. Giovanni Parmigiani for their constructive comments and suggestions.


  • Bayesian hierarchical model
  • Diagnostic test
  • Generalized linear mixed model
  • Gold standard
  • Meta-analysis
  • Missing data


Dive into the research topics of 'Random effects models in a meta-analysis of the accuracy of two diagnostic tests without a gold standard'. Together they form a unique fingerprint.

Cite this