TY - JOUR
T1 - A validity study of expert judgment procedures for setting cutoff scores on high-stakes credentialing examinations using cluster analysis
AU - Violato, Claudio
AU - Marini, Anthony
AU - Lee, Curtis
N1 - Copyright:
Copyright 2008 Elsevier B.V., All rights reserved.
PY - 2003
Y1 - 2003
N2 - This study compares an expert judgment process - minimal performance levels (MPL) using the Nedelsky and Ebel procedures - for setting cutoff scores for pass/fail on licensure examinations with an empirical approach - cluster analysis. Data from all three components of the Canadian Standard Assessment in Optometry (CSAO) examinations (knowledge, clinical judgment, and clinical skills) from 243 candidates were obtained. Results indicate that for the written components of the exams employing the Nedelsky method of MPL setting, there was a mean agreement of pass/fail of 81% with the cluster analysis approach on pass/fail categorization. For the performance exams using the Ebel method, the mean agreement of pass/fail with the cluster analysis was 93%. Thus the subjective approaches to setting cutoff scores (i.e., expert judgment methods) converge with the objective method (i.e., cluster analysis) of classifying test takers in the same categories.
AB - This study compares an expert judgment process - minimal performance levels (MPL) using the Nedelsky and Ebel procedures - for setting cutoff scores for pass/fail on licensure examinations with an empirical approach - cluster analysis. Data from all three components of the Canadian Standard Assessment in Optometry (CSAO) examinations (knowledge, clinical judgment, and clinical skills) from 243 candidates were obtained. Results indicate that for the written components of the exams employing the Nedelsky method of MPL setting, there was a mean agreement of pass/fail of 81% with the cluster analysis approach on pass/fail categorization. For the performance exams using the Ebel method, the mean agreement of pass/fail with the cluster analysis was 93%. Thus the subjective approaches to setting cutoff scores (i.e., expert judgment methods) converge with the objective method (i.e., cluster analysis) of classifying test takers in the same categories.
KW - Cluster analysis
KW - Criterion-referenced testing
KW - High-stakes examinations
KW - Minimum performance levels
UR - http://www.scopus.com/inward/record.url?scp=0037273261&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=0037273261&partnerID=8YFLogxK
U2 - 10.1177/0163278702250082
DO - 10.1177/0163278702250082
M3 - Article
C2 - 12629922
AN - SCOPUS:0037273261
SN - 0163-2787
VL - 26
SP - 59
EP - 72
JO - Evaluation and the Health Professions
JF - Evaluation and the Health Professions
IS - 1
ER -