Quantifying the Distorting Effect of Rapid Guessing on Estimates of Coefficient Αlpha

Joseph A. Rios, Jiayi Deng

Research output: Contribution to journalArticlepeer-review

Abstract

An underlying threat to the validity of reliability measures is the introduction of systematic variance in examinee scores from unintended constructs that differ from those assessed. One construct-irrelevant behavior that has gained increased attention in the literature is rapid guessing (RG), which occurs when examinees answer quickly with intentional disregard for item content. To examine the degree of distortion in coefficient alpha due to RG, this study compared alpha estimates between conditions in which simulees engaged in full solution (i.e., do not engage in RG) versus partial RG behavior. This was done by conducting a simulation study in which the percentage and ability characteristics of rapid responders as well as the percentage and pattern of RG were manipulated. After controlling for test length and difficulty, the average degree of distortion in estimates of coefficient alpha due to RG ranged from −.04 to.02 across 144 conditions. Although slight differences were noted between conditions differing in RG pattern and RG responder ability, the findings from this study suggest that estimates of coefficient alpha are largely robust to the presence of RG due to cognitive fatigue and a low perceived probability of success.

Original languageEnglish (US)
Pages (from-to)40-52
Number of pages13
JournalApplied Psychological Measurement
Volume46
Issue number1
DOIs
StatePublished - Jan 2022

Bibliographical note

Publisher Copyright:
© The Author(s) 2021.

Keywords

  • coefficient alpha
  • non-effortful responding
  • rapid guessing
  • Reliability
  • test-taking motivation

PubMed: MeSH publication types

  • Journal Article

Fingerprint

Dive into the research topics of 'Quantifying the Distorting Effect of Rapid Guessing on Estimates of Coefficient Αlpha'. Together they form a unique fingerprint.

Cite this