The effects of range restriction on estimates of criterion interrater reliability: Implications for validation research

Paul R. Sackett, Roxanne M. Laczo, Richard D. Arvey

Research output: Contribution to journalArticlepeer-review

37 Scopus citations

Abstract

This paper explores the implications of the fact that range restriction affects reliability coefficients as well as validity coefficients. Of concern was the possibility that, if range restriction produced an underestimate of criterion reliability, the use of such an estimate in correcting observed validity coefficients for attenuation could result in overestimating population validity. To address this issue, 3 scenarios were considered: truncation on the predictor, truncation on an unobserved variable, and truncation on the criterion. The 3 scenarios were found to have substantial differences in their consequences for reliability and validity estimation. Although range restriction did result in an underestimate of criterion reliability, it is generally not the case that these underestimates will result in overestimates of validity if researchers make effective use of available information about the presence and degree of range restriction.

Original languageEnglish (US)
Pages (from-to)807-825
Number of pages19
JournalPersonnel Psychology
Volume55
Issue number4
DOIs
StatePublished - Jan 1 2002

Fingerprint

Dive into the research topics of 'The effects of range restriction on estimates of criterion interrater reliability: Implications for validation research'. Together they form a unique fingerprint.

Cite this