TY - JOUR
T1 - Exploratory Study of Individual Assessment Practices
T2 - Interrater Reliability and Judgments of Assessor Effectiveness
AU - Ryan, Ann Marie
AU - Sackett, Paul R.
PY - 1989/8
Y1 - 1989/8
N2 - Three psychologists who regularly conduct individual assessments were asked to assess 3 individuals posing as job candidates for the same position. The materials from these 9 assessments-test scores, biographical information, and audiotapes of interviews-served as protocols for 50 industrial/organizational psychologists who rated the candidates and assessors. Comparisons of the approaches and conclusions of the assessors indicated variability in job/organizational information obtained, test instruments used, personal history information gathered, interview and the report generated, and conclusions regarding the candidates. On average, only one third of the raters agreed with the conclusions of the assessor whose protocol they were reviewing. Significant differences were found in the raters' evaluations of 2 of the assessors, depending on which candidate the assessor had evaluated. The study's design limits generalizability; however, the low interrater agreement is disturbing.
AB - Three psychologists who regularly conduct individual assessments were asked to assess 3 individuals posing as job candidates for the same position. The materials from these 9 assessments-test scores, biographical information, and audiotapes of interviews-served as protocols for 50 industrial/organizational psychologists who rated the candidates and assessors. Comparisons of the approaches and conclusions of the assessors indicated variability in job/organizational information obtained, test instruments used, personal history information gathered, interview and the report generated, and conclusions regarding the candidates. On average, only one third of the raters agreed with the conclusions of the assessor whose protocol they were reviewing. Significant differences were found in the raters' evaluations of 2 of the assessors, depending on which candidate the assessor had evaluated. The study's design limits generalizability; however, the low interrater agreement is disturbing.
UR - http://www.scopus.com/inward/record.url?scp=0000432569&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=0000432569&partnerID=8YFLogxK
U2 - 10.1037/0021-9010.74.4.568
DO - 10.1037/0021-9010.74.4.568
M3 - Article
AN - SCOPUS:0000432569
SN - 0021-9010
VL - 74
SP - 568
EP - 579
JO - Journal of Applied Psychology
JF - Journal of Applied Psychology
IS - 4
ER -