Changing abilities vs. changing tasks: Examining validity degradation with test scores and college performance criteria both assessed longitudinally

Jeffrey A. Dahlke, Jack W. Kostal, Paul R Sackett, Nathan R Kuncel

Research output: Contribution to journalArticlepeer-review

5 Scopus citations

Abstract

We explore potential explanations for validity degradation using a unique predictive validation data set containing up to four consecutive years of high school students' cognitive test scores and four complete years of those students' college grades. This data set permits analyses that disentangle the effects of predictor-score age and timing of criterion measurements on validity degradation. We investigate the extent to which validity degradation is explained by criterion dynamism versus the limited shelf-life of ability scores. We also explore whether validity degradation is attributable to fluctuations in criterion variability over time and/or GPA contamination from individual differences in course-taking patterns. Analyses of multiyear predictor data suggest that changes to the determinants of performance over time have much stronger effects on validity degradation than does the shelf-life of cognitive test scores. The age of predictor scores had only a modest relationship with criterion-related validity when the criterion measurement occasion was held constant. Practical implications and recommendations for future research are discussed.

Original languageEnglish (US)
Pages (from-to)980-1000
Number of pages21
JournalJournal of Applied Psychology
Volume103
Issue number9
DOIs
StatePublished - Sep 2018

Bibliographical note

Funding Information:
This research was supported by a grant from the College Board to Paul R. Sackett and Nathan R. Kuncel. Paul R. Sackett has served as a consultant to the College Board. This relationship has been reviewed and managed by the University of Minnesota in accordance with its conflict of interest policies. This research is derived from data provided by the College Board. Copyright © 2006–2013 The College Board. www.collegeboard.com.

Publisher Copyright:
© 2018 American Psychological Association.

Keywords

  • Cognitive ability
  • College performance
  • Dynamic criteria
  • Validity
  • Validity degradation

Fingerprint

Dive into the research topics of 'Changing abilities vs. changing tasks: Examining validity degradation with test scores and college performance criteria both assessed longitudinally'. Together they form a unique fingerprint.

Cite this