An investigation of item, examinee, and country correlates of rapid guessing in PISA

Joseph A. Rios, James Soland

Research output: Contribution to journalArticlepeer-review

Abstract

The objective of the present study was to investigate item-, examinee-, and country-level correlates of rapid guessing (RG) in the context of the 2018 PISA science assessment. Analyzing data from 267,148 examinees across 71 countries showed that over 50% of examinees engaged in RG on an average proportion of one in 10 items. Descriptive differences were noted between countries on the mean number of RG responses per examinee with discrepancies as large as 500%. Country-level differences in the odds of engaging in RG were associated with mean performance and regional membership. Furthermore, based on a two-level cross-classified hierarchical linear model, both item- and examinee-level correlates were found to moderate the likelihood of RG. Specifically, the inclusion of items with multimedia content was associated with a decrease in RG. A number of demographic and attitudinal examinee-level variables were also significant moderators, including sex, linguistic background, SES, and self-rated reading comprehension, motivation mastery, and fear of failure. The findings from this study imply that select subgroup comparisons within and across nations may be biased by differential test-taking effort. To mitigate RG in international assessments, future test developers may look to leverage technology-enhanced items.

Original languageEnglish (US)
Pages (from-to)154-184
Number of pages31
JournalInternational Journal of Testing
Volume22
Issue number2
DOIs
StatePublished - 2022

Bibliographical note

Publisher Copyright:
© 2022 International Test Commission.

Keywords

  • Rapid guessing
  • effort
  • international assessments
  • test-taking

Fingerprint

Dive into the research topics of 'An investigation of item, examinee, and country correlates of rapid guessing in PISA'. Together they form a unique fingerprint.

Cite this