An Examination of Individual Ability Estimation and Classification Accuracy Under Rapid Guessing Misidentifications

Joseph Rios

Research output: Contribution to journalArticlepeer-review

Abstract

To mitigate the deleterious effects of rapid guessing (RG) on ability estimates, several rescoring procedures have been proposed. Underlying many of these procedures is the assumption that RG is accurately identified. At present, there have been minimal investigations examining the utility of rescoring approaches when RG is misclassified, and individual scores are reported. To address this limitation, the present simulation study investigates the effect of RG misclassifications on individual examinee ability estimate bias and classification accuracy when using effort-moderated (EM) scoring. This objective is accomplished by manipulating simulee ability level, RG rate, as well as misclassification type and percentage. Results showed that EM scoring significantly improved ability inferences for examinees engaging in RG; however, the effectiveness of this approach was largely dependent on misclassification type. Specifically, across ability levels, bias tended to be on average lower when falsely classifying effortful responses as RG. Although EM scoring improved bias, it was susceptible to elevated false-positive classifications of ability under high RG.

Original languageEnglish (US)
Pages (from-to)300-312
Number of pages13
JournalApplied Measurement in Education
Volume35
Issue number4
DOIs
StatePublished - 2022

Bibliographical note

Publisher Copyright:
© 2022 Taylor & Francis Group, LLC.

Fingerprint

Dive into the research topics of 'An Examination of Individual Ability Estimation and Classification Accuracy Under Rapid Guessing Misidentifications'. Together they form a unique fingerprint.

Cite this