We explore potential explanations for validity degradation using a unique predictive validation data set containing up to four consecutive years of high school students' cognitive test scores and four complete years of those students' college grades. This data set permits analyses that disentangle the effects of predictor-score age and timing of criterion measurements on validity degradation. We investigate the extent to which validity degradation is explained by criterion dynamism versus the limited shelf-life of ability scores. We also explore whether validity degradation is attributable to fluctuations in criterion variability over time and/or GPA contamination from individual differences in course-taking patterns. Analyses of multiyear predictor data suggest that changes to the determinants of performance over time have much stronger effects on validity degradation than does the shelf-life of cognitive test scores. The age of predictor scores had only a modest relationship with criterion-related validity when the criterion measurement occasion was held constant. Practical implications and recommendations for future research are discussed.
Bibliographical noteFunding Information:
This research was supported by a grant from the College Board to Paul R. Sackett and Nathan R. Kuncel. Paul R. Sackett has served as a consultant to the College Board. This relationship has been reviewed and managed by the University of Minnesota in accordance with its conflict of interest policies. This research is derived from data provided by the College Board. Copyright © 2006–2013 The College Board. www.collegeboard.com.
© 2018 American Psychological Association.
- Cognitive ability
- College performance
- Dynamic criteria
- Validity degradation