When Should Individual Ability Estimates Be Reported if Rapid Guessing Is Present?

Joseph A. Rios

Research output: Contribution to journalArticlepeer-review

1 Scopus citations

Abstract

Testing programs are confronted with the decision of whether to report individual scores for examinees that have engaged in rapid guessing (RG). As noted by the Standards for Educational and Psychological Testing, this decision should be based on a documented criterion that determines score exclusion. To this end, a number of heuristic criteria (e.g., exclude all examinees with RG rates of 10%) have been adopted in the literature. Given that these criteria lack strong methodological support, the objective of this simulation study was to evaluate their appropriateness in terms of individual ability estimate and classification accuracy when manipulating both assessment and RG characteristics. The findings provide evidence that employing a common criterion for all examinees may be an ineffective strategy because a given RG percentage may have differing degrees of biasing effects based on test difficulty, examinee ability, and RG pattern. These results suggest that practitioners may benefit from establishing context-specific exclusion criteria that consider test purpose, score use, and targeted examinee trait levels.

Original languageEnglish (US)
Pages (from-to)222-236
Number of pages15
JournalApplied Measurement in Education
Volume35
Issue number3
DOIs
StatePublished - 2022

Bibliographical note

Publisher Copyright:
© 2022 Taylor & Francis Group, LLC.

Fingerprint

Dive into the research topics of 'When Should Individual Ability Estimates Be Reported if Rapid Guessing Is Present?'. Together they form a unique fingerprint.

Cite this