Performance of between-study heterogeneity measures in the cochrane library

The growth in comparative effectiveness research and evidence-based medicine has increased attention to systematic reviews and meta-analyses. Meta-analysis synthesizes and contrasts evidence from multiple independent studies to improve statistical efficiency and reduce bias. Assessing heterogeneity is critical for performing a meta-analysis and interpreting results. As a widely used heterogeneity measure, the I² statistic quantifies the proportion of total variation across studies that is caused by real differences in effect size. The presence of outlying studies can seriously exaggerate the I² statistic. Two alternative heterogeneity measures, the I_r² and I_m² , have been recently proposed to reduce the impact of outlying studies. To evaluate these measures' performance empirically, we applied them to 20,599 meta-analyses in the Cochrane Library. We found that the I_r² and I_m² have strong agreement with the I², while they are more robust than the I² when outlying studies appear.