Effects of range restriction and criterion contamination on differential validity of the SAT by race/ethnicity and sex

Jeffrey A. Dahlke, Paul R Sackett, Nathan R Kuncel

Research output: Contribution to journalArticle

Abstract

We illustrate the effects of range restriction and a form of criterion contamination (individual differences in course-taking patterns) on the validity of SAT scores for predicting college academic performance. College data facilitate exploration of differential validity's determinants because they (a) permit the use multivariate range-restriction corrections to more accurately account for differential range restriction across subgroups and (b) allow for separate examinations of composite performance and specific performance episodes, the latter of which controls for ecological contamination of composite performance due to individuals' choices of performance opportunities. Using data from 363,004 students at 107 U.S. institutions, we found that controlling for course-taking patterns resulted in validity coefficients that were appreciably larger than predictors' correlations with obtained grade point averages (GPAs). The validities of SAT scores for predicting the first-year college performance of Black and Hispanic students were not significantly different from the validity for White students after correcting for both coursetaking patterns and differential range restriction, but significant Black-White differences were detected for predicting 4-year cumulative performance. Validity estimates for predicting both first-year and 4-year cumulative performance were significantly smaller among Asian students than White students after making these corrections. The SAT's observed validity for predicting college GPAs was substantially lower for males than females and, unexpectedly, controlling for course-taking patterns increased male-female validity differences. Implications for personnel selection research are discussed.

Original languageEnglish (US)
Pages (from-to)814-831
Number of pages18
JournalJournal of Applied Psychology
Volume104
Issue number6
DOIs
StatePublished - Jun 1 2019

Fingerprint

Students
Personnel Selection
Hispanic Americans
Individuality
Research

Keywords

  • Cognitive ability
  • Criterion contamination
  • Differential validity
  • Range restriction
  • Standardized testing

PubMed: MeSH publication types

  • Journal Article

Cite this

@article{4d19f10e1e824f2aaafb3f0235fc6f5c,
title = "Effects of range restriction and criterion contamination on differential validity of the SAT by race/ethnicity and sex",
abstract = "We illustrate the effects of range restriction and a form of criterion contamination (individual differences in course-taking patterns) on the validity of SAT scores for predicting college academic performance. College data facilitate exploration of differential validity's determinants because they (a) permit the use multivariate range-restriction corrections to more accurately account for differential range restriction across subgroups and (b) allow for separate examinations of composite performance and specific performance episodes, the latter of which controls for ecological contamination of composite performance due to individuals' choices of performance opportunities. Using data from 363,004 students at 107 U.S. institutions, we found that controlling for course-taking patterns resulted in validity coefficients that were appreciably larger than predictors' correlations with obtained grade point averages (GPAs). The validities of SAT scores for predicting the first-year college performance of Black and Hispanic students were not significantly different from the validity for White students after correcting for both coursetaking patterns and differential range restriction, but significant Black-White differences were detected for predicting 4-year cumulative performance. Validity estimates for predicting both first-year and 4-year cumulative performance were significantly smaller among Asian students than White students after making these corrections. The SAT's observed validity for predicting college GPAs was substantially lower for males than females and, unexpectedly, controlling for course-taking patterns increased male-female validity differences. Implications for personnel selection research are discussed.",
keywords = "Cognitive ability, Criterion contamination, Differential validity, Range restriction, Standardized testing",
author = "Dahlke, {Jeffrey A.} and Sackett, {Paul R} and Kuncel, {Nathan R}",
year = "2019",
month = "6",
day = "1",
doi = "10.1037/apl0000382",
language = "English (US)",
volume = "104",
pages = "814--831",
journal = "Journal of Applied Psychology",
issn = "0021-9010",
publisher = "American Psychological Association",
number = "6",

}

TY - JOUR

T1 - Effects of range restriction and criterion contamination on differential validity of the SAT by race/ethnicity and sex

AU - Dahlke, Jeffrey A.

AU - Sackett, Paul R

AU - Kuncel, Nathan R

PY - 2019/6/1

Y1 - 2019/6/1

N2 - We illustrate the effects of range restriction and a form of criterion contamination (individual differences in course-taking patterns) on the validity of SAT scores for predicting college academic performance. College data facilitate exploration of differential validity's determinants because they (a) permit the use multivariate range-restriction corrections to more accurately account for differential range restriction across subgroups and (b) allow for separate examinations of composite performance and specific performance episodes, the latter of which controls for ecological contamination of composite performance due to individuals' choices of performance opportunities. Using data from 363,004 students at 107 U.S. institutions, we found that controlling for course-taking patterns resulted in validity coefficients that were appreciably larger than predictors' correlations with obtained grade point averages (GPAs). The validities of SAT scores for predicting the first-year college performance of Black and Hispanic students were not significantly different from the validity for White students after correcting for both coursetaking patterns and differential range restriction, but significant Black-White differences were detected for predicting 4-year cumulative performance. Validity estimates for predicting both first-year and 4-year cumulative performance were significantly smaller among Asian students than White students after making these corrections. The SAT's observed validity for predicting college GPAs was substantially lower for males than females and, unexpectedly, controlling for course-taking patterns increased male-female validity differences. Implications for personnel selection research are discussed.

AB - We illustrate the effects of range restriction and a form of criterion contamination (individual differences in course-taking patterns) on the validity of SAT scores for predicting college academic performance. College data facilitate exploration of differential validity's determinants because they (a) permit the use multivariate range-restriction corrections to more accurately account for differential range restriction across subgroups and (b) allow for separate examinations of composite performance and specific performance episodes, the latter of which controls for ecological contamination of composite performance due to individuals' choices of performance opportunities. Using data from 363,004 students at 107 U.S. institutions, we found that controlling for course-taking patterns resulted in validity coefficients that were appreciably larger than predictors' correlations with obtained grade point averages (GPAs). The validities of SAT scores for predicting the first-year college performance of Black and Hispanic students were not significantly different from the validity for White students after correcting for both coursetaking patterns and differential range restriction, but significant Black-White differences were detected for predicting 4-year cumulative performance. Validity estimates for predicting both first-year and 4-year cumulative performance were significantly smaller among Asian students than White students after making these corrections. The SAT's observed validity for predicting college GPAs was substantially lower for males than females and, unexpectedly, controlling for course-taking patterns increased male-female validity differences. Implications for personnel selection research are discussed.

KW - Cognitive ability

KW - Criterion contamination

KW - Differential validity

KW - Range restriction

KW - Standardized testing

UR - http://www.scopus.com/inward/record.url?scp=85065982868&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85065982868&partnerID=8YFLogxK

U2 - 10.1037/apl0000382

DO - 10.1037/apl0000382

M3 - Article

C2 - 30640487

AN - SCOPUS:85065982868

VL - 104

SP - 814

EP - 831

JO - Journal of Applied Psychology

JF - Journal of Applied Psychology

SN - 0021-9010

IS - 6

ER -