Constructing Subscores That Add Validity: A Case Study of Identifying Students at Risk

Gina Biancarosa, Patrick C. Kennedy, Sarah E. Carlson, Hyeon Jin Yoon, Ben Seipel, Bowen Liu, Mark L Davison

Research output: Contribution to journalArticlepeer-review

5 Scopus citations

Abstract

Prior research suggests that subscores from a single achievement test seldom add value over a single total score. Such scores typically correspond to subcontent areas in the total content domain, but content subdomains might not provide a sound basis for subscores. Using scores on an inferential reading comprehension test from 625 third, fourth, and fifth graders, two new methods of creating subscores were explored. Three subscores were based on the types of incorrect answers given by students. The fourth was based on temporal efficiency in giving correct answers. All four scores were reliable. The three subscores based on incorrect answers added value and validity. In logistic regression analyses predicting failure to reach proficiency on a statewide test, models including subscores fit better than the model with a single total score. Including the pattern of incorrect responses improved fit in all three grades, whereas including the comprehension efficiency score only modestly improved fit in fourth and fifth grades, but not third grade. Area under the curve (AUC) statistics from receiver operating characteristic (ROC) curves based on the various models were higher for models including subscores than those without subscores. Implications for using models with and without subscores are illustrated and discussed.

Original languageEnglish (US)
Pages (from-to)65-84
Number of pages20
JournalEducational and Psychological Measurement
Volume79
Issue number1
DOIs
StatePublished - Feb 1 2019

Bibliographical note

Funding Information:
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305A140185 to the University of Oregon.

Publisher Copyright:
© The Author(s) 2018.

Keywords

  • at-risk screening
  • diagnostic testing
  • formative assessment
  • reading comprehension
  • subscores

PubMed: MeSH publication types

  • Journal Article

Fingerprint

Dive into the research topics of 'Constructing Subscores That Add Validity: A Case Study of Identifying Students at Risk'. Together they form a unique fingerprint.

Cite this