Influence of Selected-Response Format Variants on Test Characteristics and Test-Taking Effort: An Empirical Study

Hongwen Guo, Joseph A. Rios, Guangming Ling, Zhen Wang, Lin Gu, Zhitong Yang, Lydia O. Liu

Research output: Contribution to journalArticlepeer-review

1 Scopus citations

Abstract

Different variants of the selected-response (SR) item type have been developed for various reasons (i.e., simulating realistic situations, examining critical-thinking and/or problem-solving skills). Generally, the variants of SR item format are more complex than the traditional multiple-choice (MC) items, which may be more challenging to test takers and thus may discourage their test engagement on low-stakes assessments. Low test-taking effort has been shown to distort test scores and thereby diminish score validity. We used data collected from a large-scale assessment to investigate how variants of the SR item format may impact test properties and test engagement. Results show that the studied variants of SR item format were generally harder and more time consuming compared to the traditional MC item format, but they did not show negative impact on test-taking effort. However, item position had a dominating influence on nonresponse rates and rapid-guessing rates in a cumulative fashion, even though the effect sizes were relatively small in the studied data.

Original languageEnglish (US)
Pages (from-to)1-20
Number of pages20
JournalETS Research Report Series
Volume2022
Issue number1
DOIs
StatePublished - Dec 2022

Bibliographical note

Publisher Copyright:
© 2022 Educational Testing Service.

Keywords

  • New item type
  • low-stakes assessment
  • test-taking effort

Fingerprint

Dive into the research topics of 'Influence of Selected-Response Format Variants on Test Characteristics and Test-Taking Effort: An Empirical Study'. Together they form a unique fingerprint.

Cite this