If numerical reasoning items are administered under time limits, will two dimensions be required to account for the responses, a numerical ability dimension and a speed dimension? A total of 182 college students answered 74 numerical reasoning items. Every item was taken with and without time limits by half the students. Three psychometric models were fit to the data-one including no time-limit effect, one including a fixed time-limit effect, and one including a random effect of time limits. The latter model best fit the data, suggesting that a speed dimension, the random effect of time limits, is needed to account for time-limited responses. The estimated reliability of the Speed scores was .39. Despite this low reliability, Speed scores were correlated with American College Testing (ACT) math scores and response times. Speed scores added significantly to the ACT math score variance accounted for by the numerical reasoning dimension in the model. A within-person log-odds ratio interpretation of the Speed score is proposed. Possible methods of improving Speed score reliability and methods for studying the speed dimension are discussed.
Bibliographical noteFunding Information:
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by a United States Army Research Institute for the Behavioral and Social Sciences contract award number 1435-04-03-CT-74083 (Mark L. Davison) and by Grant No. R305C050059 from the Institute of Education Sciences in the U.S. Department of Education. The rights of research participants were protected and applicable human research guidelines were followed.
- Rasch model
- hierarchical linear model
- linear logistic model
- multidimensional item-response theory
- numerical reasoning
- quantitative reasoning
- random effects
- response time limits
- test time limits