Assessing the Accuracy of Parameter Estimates in the Presence of Rapid Guessing Misclassifications

Joseph A. Rios

Research output: Contribution to journalArticlepeer-review

13 Scopus citations


The presence of rapid guessing (RG) presents a challenge to practitioners in obtaining accurate estimates of measurement properties and examinee ability. In response to this concern, researchers have utilized response times as a proxy of RG and have attempted to improve parameter estimation accuracy by filtering RG responses using popular scoring approaches, such as the effort-moderated item response theory (EM-IRT) model. However, such an approach assumes that RG can be correctly identified based on an indirect proxy of examinee behavior. A failure to meet this assumption leads to the inclusion of distortive and psychometrically uninformative information in parameter estimates. To address this issue, a simulation study was conducted to examine how violations to the assumption of correct RG classification influences EM-IRT item and ability parameter estimation accuracy and compares these results with parameter estimates from the three-parameter logistic (3PL) model, which includes RG responses in scoring. Two RG misclassification factors were manipulated: type (underclassification vs. overclassification) and rate (10%, 30%, and 50%). Results indicated that the EM-IRT model provided improved item parameter estimation over the 3PL model regardless of misclassification type and rate. Furthermore, under most conditions, increased rates of RG underclassification were associated with the greatest bias in ability parameter estimates from the EM-IRT model. In spite of this, the EM-IRT model with RG misclassifications demonstrated more accurate ability parameter estimation than the 3PL model when the mean ability of RG subgroups did not differ. This suggests that in certain situations it may be better for practitioners to (a) imperfectly identify RG than to ignore the presence of such invalid responses and (b) select liberal over conservative response time thresholds to mitigate bias from underclassified RG.

Original languageEnglish (US)
Pages (from-to)122-150
Number of pages29
JournalEducational and Psychological Measurement
Issue number1
StateAccepted/In press - 2021

Bibliographical note

Funding Information:
The author would like to thank Samuel Ihlenfeldt from the University of Minnesota for his helpful comments on an earlier draft. The author(s) received no financial support for the research, authorship, and/or publication of this article.

Publisher Copyright:
© The Author(s) 2021.


  • IRT
  • item response theory
  • noneffortful responding
  • parameter estimation
  • rapid guessing
  • response times


Dive into the research topics of 'Assessing the Accuracy of Parameter Estimates in the Presence of Rapid Guessing Misclassifications'. Together they form a unique fingerprint.

Cite this