Improving the Predictive Validity of Reading Comprehension Using Response Times of Correct Item Responses

Shiyang Su, Mark L Davison

Research output: Contribution to journalArticle

Abstract

Response times have often been used as ancillary information to improve parameter estimation. Under the dual processing theory, assuming reading comprehension requires an automatic process, a fast, correct response is an indicator of effective automatic processing. A skilled, automatic comprehender should be high in response accuracy and low in response times. Following this argument, several questions were addressed in this study. First, individuals with higher ability endorsed a correct response more quickly in the reading comprehension assessment, suggesting correct response times provide useful information in discriminating individuals of different abilities. Second, in terms of predicting an external criterion of reading proficiency, the increment of predictive validity of categorized correct response times was larger than the incremental validity of continuous response times based on both correct and incorrect response times. An index reflecting both accuracy and response time yielded higher incremental validity than indices reflecting response time only. Results support response time as manifesting a dimension of interest in its own right, over its inclusion as an ancillary dimension in a multidimensional model.

Original languageEnglish (US)
Pages (from-to)166-182
Number of pages17
JournalApplied Measurement in Education
Volume32
Issue number2
DOIs
StatePublished - Apr 3 2019

Fingerprint

Reaction Time
Reading
comprehension
Aptitude
time
ability
inclusion

Cite this

Improving the Predictive Validity of Reading Comprehension Using Response Times of Correct Item Responses. / Su, Shiyang; Davison, Mark L.

In: Applied Measurement in Education, Vol. 32, No. 2, 03.04.2019, p. 166-182.

Research output: Contribution to journalArticle

@article{5643e5d3ee7b4a8488819e458c559ca1,
title = "Improving the Predictive Validity of Reading Comprehension Using Response Times of Correct Item Responses",
abstract = "Response times have often been used as ancillary information to improve parameter estimation. Under the dual processing theory, assuming reading comprehension requires an automatic process, a fast, correct response is an indicator of effective automatic processing. A skilled, automatic comprehender should be high in response accuracy and low in response times. Following this argument, several questions were addressed in this study. First, individuals with higher ability endorsed a correct response more quickly in the reading comprehension assessment, suggesting correct response times provide useful information in discriminating individuals of different abilities. Second, in terms of predicting an external criterion of reading proficiency, the increment of predictive validity of categorized correct response times was larger than the incremental validity of continuous response times based on both correct and incorrect response times. An index reflecting both accuracy and response time yielded higher incremental validity than indices reflecting response time only. Results support response time as manifesting a dimension of interest in its own right, over its inclusion as an ancillary dimension in a multidimensional model.",
author = "Shiyang Su and Davison, {Mark L}",
year = "2019",
month = "4",
day = "3",
doi = "10.1080/08957347.2019.1577247",
language = "English (US)",
volume = "32",
pages = "166--182",
journal = "Applied Measurement in Education",
issn = "0895-7347",
publisher = "Routledge",
number = "2",

}

TY - JOUR

T1 - Improving the Predictive Validity of Reading Comprehension Using Response Times of Correct Item Responses

AU - Su, Shiyang

AU - Davison, Mark L

PY - 2019/4/3

Y1 - 2019/4/3

N2 - Response times have often been used as ancillary information to improve parameter estimation. Under the dual processing theory, assuming reading comprehension requires an automatic process, a fast, correct response is an indicator of effective automatic processing. A skilled, automatic comprehender should be high in response accuracy and low in response times. Following this argument, several questions were addressed in this study. First, individuals with higher ability endorsed a correct response more quickly in the reading comprehension assessment, suggesting correct response times provide useful information in discriminating individuals of different abilities. Second, in terms of predicting an external criterion of reading proficiency, the increment of predictive validity of categorized correct response times was larger than the incremental validity of continuous response times based on both correct and incorrect response times. An index reflecting both accuracy and response time yielded higher incremental validity than indices reflecting response time only. Results support response time as manifesting a dimension of interest in its own right, over its inclusion as an ancillary dimension in a multidimensional model.

AB - Response times have often been used as ancillary information to improve parameter estimation. Under the dual processing theory, assuming reading comprehension requires an automatic process, a fast, correct response is an indicator of effective automatic processing. A skilled, automatic comprehender should be high in response accuracy and low in response times. Following this argument, several questions were addressed in this study. First, individuals with higher ability endorsed a correct response more quickly in the reading comprehension assessment, suggesting correct response times provide useful information in discriminating individuals of different abilities. Second, in terms of predicting an external criterion of reading proficiency, the increment of predictive validity of categorized correct response times was larger than the incremental validity of continuous response times based on both correct and incorrect response times. An index reflecting both accuracy and response time yielded higher incremental validity than indices reflecting response time only. Results support response time as manifesting a dimension of interest in its own right, over its inclusion as an ancillary dimension in a multidimensional model.

UR - http://www.scopus.com/inward/record.url?scp=85062922141&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85062922141&partnerID=8YFLogxK

U2 - 10.1080/08957347.2019.1577247

DO - 10.1080/08957347.2019.1577247

M3 - Article

VL - 32

SP - 166

EP - 182

JO - Applied Measurement in Education

JF - Applied Measurement in Education

SN - 0895-7347

IS - 2

ER -