Improving the Meta-Analytic Assessment of Effect Size Variance With an Informed Bayesian Prior

Piers Steel, John Kammeyer-Mueller, Ted A. Paterson

Research output: Contribution to journalArticlepeer-review

14 Scopus citations

Abstract

Meta-analytic estimation of effect size variance is critical for determining the degree to which a relationship or finding generalizes across contexts. In most meta-analyses, population effect size variability is estimated by subtracting expected sampling error variance from observed variance, using only information from a limited set of available studies. We propose an improved Bayesian variance estimation technique that incorporates findings from previous meta-analytic research through an informed prior distribution of likely levels of effect size variance. The logic of exchangeability as a conceptual foundation for using an informed prior is explicated. On the basis of Monte Carlo simulations, we find the traditional method of meta-analytic variance estimation the most biased and least accurate technique across all sizes of meta-analyses considered. The Bayesian methodology incorporating an informed prior proved to be the most accurate and overall least biased of all estimation methods. Conceptual advantages and limitations that must be taken into account when incorporating an informed prior to estimate variability of effect sizes in a meta-analysis are also discussed.

Original languageEnglish (US)
Pages (from-to)718-743
Number of pages26
JournalJournal of Management
Volume41
Issue number2
DOIs
StatePublished - Feb 27 2015

Keywords

  • Bayesian statistics
  • effect size variance
  • meta-analysis
  • simulations

Fingerprint Dive into the research topics of 'Improving the Meta-Analytic Assessment of Effect Size Variance With an Informed Bayesian Prior'. Together they form a unique fingerprint.

Cite this