A validity study of expert judgment procedures for setting cutoff scores on high-stakes credentialing examinations using cluster analysis

Claudio Violato, Anthony Marini, Curtis Lee

Research output: Contribution to journalArticlepeer-review

15 Scopus citations

Abstract

This study compares an expert judgment process - minimal performance levels (MPL) using the Nedelsky and Ebel procedures - for setting cutoff scores for pass/fail on licensure examinations with an empirical approach - cluster analysis. Data from all three components of the Canadian Standard Assessment in Optometry (CSAO) examinations (knowledge, clinical judgment, and clinical skills) from 243 candidates were obtained. Results indicate that for the written components of the exams employing the Nedelsky method of MPL setting, there was a mean agreement of pass/fail of 81% with the cluster analysis approach on pass/fail categorization. For the performance exams using the Ebel method, the mean agreement of pass/fail with the cluster analysis was 93%. Thus the subjective approaches to setting cutoff scores (i.e., expert judgment methods) converge with the objective method (i.e., cluster analysis) of classifying test takers in the same categories.

Original languageEnglish (US)
Pages (from-to)59-72
Number of pages14
JournalEvaluation and the Health Professions
Volume26
Issue number1
DOIs
StatePublished - 2003

Bibliographical note

Copyright:
Copyright 2008 Elsevier B.V., All rights reserved.

Keywords

  • Cluster analysis
  • Criterion-referenced testing
  • High-stakes examinations
  • Minimum performance levels

Fingerprint

Dive into the research topics of 'A validity study of expert judgment procedures for setting cutoff scores on high-stakes credentialing examinations using cluster analysis'. Together they form a unique fingerprint.

Cite this