A comment on sampling error in the standardized mean difference with unequal sample sizes

Avoiding potential errors in meta-analytic and primary research

Roxanne M. Laczo, Paul R Sackett, Philip Bobko, José M. Cortina

Research output: Contribution to journalReview article

10 Citations (Scopus)

Abstract

The authors discuss potential confusion in conducting primary studies and meta-analyses on the basis of differences between groups. First, the authors show that a formula for the sampling error of the standardized mean difference (d) that is based on equal group sample sizes can produce substantially biased results if applied with markedly unequal group sizes. Second, the authors show that the same concerns are present when primary analyses or meta-analyses are conducted with point-biserial correlations, as the point-biserial correlation (r) is a transformation of d. Third, the authors examine the practice of correcting a point-biserial r for unequal sample sizes and note that such correction would also increase the sampling error of the corrected r. Correcting rs for unequal sample sizes, but using the standard formula for sampling error in uncorrected r, can result in bias. The authors offer a set of recommendations for conducting meta-analyses of group differences.

Original languageEnglish (US)
Pages (from-to)758-764
Number of pages7
JournalJournal of Applied Psychology
Volume90
Issue number4
DOIs
StatePublished - Jul 1 2005

Fingerprint

Selection Bias
Sample Size
Meta-Analysis
Research

Keywords

  • Effect size
  • Meta-analysis
  • Sampling error
  • Subgroup proportion

Cite this

A comment on sampling error in the standardized mean difference with unequal sample sizes : Avoiding potential errors in meta-analytic and primary research. / Laczo, Roxanne M.; Sackett, Paul R; Bobko, Philip; Cortina, José M.

In: Journal of Applied Psychology, Vol. 90, No. 4, 01.07.2005, p. 758-764.

Research output: Contribution to journalReview article

@article{24fa88efd6864b7dbd5a4c5cf10cbd87,
title = "A comment on sampling error in the standardized mean difference with unequal sample sizes: Avoiding potential errors in meta-analytic and primary research",
abstract = "The authors discuss potential confusion in conducting primary studies and meta-analyses on the basis of differences between groups. First, the authors show that a formula for the sampling error of the standardized mean difference (d) that is based on equal group sample sizes can produce substantially biased results if applied with markedly unequal group sizes. Second, the authors show that the same concerns are present when primary analyses or meta-analyses are conducted with point-biserial correlations, as the point-biserial correlation (r) is a transformation of d. Third, the authors examine the practice of correcting a point-biserial r for unequal sample sizes and note that such correction would also increase the sampling error of the corrected r. Correcting rs for unequal sample sizes, but using the standard formula for sampling error in uncorrected r, can result in bias. The authors offer a set of recommendations for conducting meta-analyses of group differences.",
keywords = "Effect size, Meta-analysis, Sampling error, Subgroup proportion",
author = "Laczo, {Roxanne M.} and Sackett, {Paul R} and Philip Bobko and Cortina, {Jos{\'e} M.}",
year = "2005",
month = "7",
day = "1",
doi = "10.1037/0021-9010.90.4.758",
language = "English (US)",
volume = "90",
pages = "758--764",
journal = "Journal of Applied Psychology",
issn = "0021-9010",
publisher = "American Psychological Association",
number = "4",

}

TY - JOUR

T1 - A comment on sampling error in the standardized mean difference with unequal sample sizes

T2 - Avoiding potential errors in meta-analytic and primary research

AU - Laczo, Roxanne M.

AU - Sackett, Paul R

AU - Bobko, Philip

AU - Cortina, José M.

PY - 2005/7/1

Y1 - 2005/7/1

N2 - The authors discuss potential confusion in conducting primary studies and meta-analyses on the basis of differences between groups. First, the authors show that a formula for the sampling error of the standardized mean difference (d) that is based on equal group sample sizes can produce substantially biased results if applied with markedly unequal group sizes. Second, the authors show that the same concerns are present when primary analyses or meta-analyses are conducted with point-biserial correlations, as the point-biserial correlation (r) is a transformation of d. Third, the authors examine the practice of correcting a point-biserial r for unequal sample sizes and note that such correction would also increase the sampling error of the corrected r. Correcting rs for unequal sample sizes, but using the standard formula for sampling error in uncorrected r, can result in bias. The authors offer a set of recommendations for conducting meta-analyses of group differences.

AB - The authors discuss potential confusion in conducting primary studies and meta-analyses on the basis of differences between groups. First, the authors show that a formula for the sampling error of the standardized mean difference (d) that is based on equal group sample sizes can produce substantially biased results if applied with markedly unequal group sizes. Second, the authors show that the same concerns are present when primary analyses or meta-analyses are conducted with point-biserial correlations, as the point-biserial correlation (r) is a transformation of d. Third, the authors examine the practice of correcting a point-biserial r for unequal sample sizes and note that such correction would also increase the sampling error of the corrected r. Correcting rs for unequal sample sizes, but using the standard formula for sampling error in uncorrected r, can result in bias. The authors offer a set of recommendations for conducting meta-analyses of group differences.

KW - Effect size

KW - Meta-analysis

KW - Sampling error

KW - Subgroup proportion

UR - http://www.scopus.com/inward/record.url?scp=24944562050&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=24944562050&partnerID=8YFLogxK

U2 - 10.1037/0021-9010.90.4.758

DO - 10.1037/0021-9010.90.4.758

M3 - Review article

VL - 90

SP - 758

EP - 764

JO - Journal of Applied Psychology

JF - Journal of Applied Psychology

SN - 0021-9010

IS - 4

ER -