Empirical Comparisons of 12 Meta-analysis Methods for Synthesizing Proportions of Binary Outcomes

Lifeng Lin, Chang Xu, Haitao Chu

Research output: Contribution to journalArticlepeer-review

8 Scopus citations


BACKGROUND: Meta-analysis is increasingly used to synthesize proportions (e.g., disease prevalence). It can be implemented with widely used two-step methods or one-step methods, such as generalized linear mixed models (GLMMs). Existing simulation studies have shown that GLMMs outperform the two-step methods in some settings. It is, however, unclear whether these simulation settings are common in the real world. We aim to compare the real-world performance of various meta-analysis methods for synthesizing proportions.

METHODS: We extracted datasets of proportions from the Cochrane Library and applied 12 two-step and one-step methods to each dataset. We used Spearman's ρ and the Bland-Altman plot to assess their results' correlation and agreement. The GLMM with the logit link was chosen as the reference method. We calculated the absolute difference and fold change (ratio of estimates) of the overall proportion estimates produced by each method vs. the reference method.

RESULTS: We obtained a total of 43,644 datasets. The various methods generally had high correlations (ρ > 0.9) and agreements. GLMMs had computational issues more frequently than the two-step methods. However, the two-step methods generally produced large absolute differences from the GLMM with the logit link for small total sample sizes (< 50) and crude event rates within 10-20% and 90-95%, and large fold changes for small total event counts (< 10) and low crude event rates (< 20%).

CONCLUSIONS: Although different methods produced similar overall proportion estimates in most datasets, one-step methods should be considered in the presence of small total event counts or sample sizes and very low or high event rates.

Original languageEnglish (US)
Pages (from-to)308-317
Number of pages10
JournalJournal of general internal medicine
Issue number2
Early online dateSep 10 2021
StatePublished - Feb 2022

Bibliographical note

Funding Information:
This research was supported in part by the US National Institutes of Health/National Library of Medicine grant R01 LM012982 and National Institutes of Health/National Center for Advancing Translational Sciences grant UL1 TR001427. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The financial support had no involvement in the conceptualization of the report and the decision to submit the report for publication.

Publisher Copyright:
© 2021, Society of General Internal Medicine.


  • Cochrane review
  • data transformation
  • generalized linear mixed model
  • meta-analysis
  • proportion

PubMed: MeSH publication types

  • Journal Article
  • Meta-Analysis
  • Research Support, N.I.H., Extramural


Dive into the research topics of 'Empirical Comparisons of 12 Meta-analysis Methods for Synthesizing Proportions of Binary Outcomes'. Together they form a unique fingerprint.

Cite this