Computerized Adaptive Testing in Early Education: Exploring the Impact of Item Position Effects on Ability Estimation

Anthony D. Albano, Liuhan Cai, Erin M. Lease, Scott R McConnell

Research output: Contribution to journalArticle

Abstract

Studies have shown that item difficulty can vary significantly based on the context of an item within a test form. In particular, item position may be associated with practice and fatigue effects that influence item parameter estimation. The purpose of this research was to examine the relevance of item position specifically for assessments used in early education, an area of testing that has received relatively limited psychometric attention. In an initial study, multilevel item response models fit to data from an early literacy measure revealed statistically significant increases in difficulty for items appearing later in a 20-item form. The estimated linear change in logits for an increase of 1 in position was.024, resulting in a predicted change of.46 logits for a shift from the beginning to the end of the form. A subsequent simulation study examined impacts of item position effects on person ability estimation within computerized adaptive testing. Implications and recommendations for practice are discussed.

Original languageEnglish (US)
Pages (from-to)437-451
Number of pages15
JournalJournal of Educational Measurement
Volume56
Issue number2
DOIs
StatePublished - Jun 1 2019

Fingerprint

Aptitude
Psychometrics
Fatigue
Education
ability
Research
fatigue
psychometrics
education
literacy
simulation
human being
Literacy

Cite this

Computerized Adaptive Testing in Early Education : Exploring the Impact of Item Position Effects on Ability Estimation. / Albano, Anthony D.; Cai, Liuhan; Lease, Erin M.; McConnell, Scott R.

In: Journal of Educational Measurement, Vol. 56, No. 2, 01.06.2019, p. 437-451.

Research output: Contribution to journalArticle

@article{c21eb0bc1dfa4ea6bf8e2d7326fbce55,
title = "Computerized Adaptive Testing in Early Education: Exploring the Impact of Item Position Effects on Ability Estimation",
abstract = "Studies have shown that item difficulty can vary significantly based on the context of an item within a test form. In particular, item position may be associated with practice and fatigue effects that influence item parameter estimation. The purpose of this research was to examine the relevance of item position specifically for assessments used in early education, an area of testing that has received relatively limited psychometric attention. In an initial study, multilevel item response models fit to data from an early literacy measure revealed statistically significant increases in difficulty for items appearing later in a 20-item form. The estimated linear change in logits for an increase of 1 in position was.024, resulting in a predicted change of.46 logits for a shift from the beginning to the end of the form. A subsequent simulation study examined impacts of item position effects on person ability estimation within computerized adaptive testing. Implications and recommendations for practice are discussed.",
author = "Albano, {Anthony D.} and Liuhan Cai and Lease, {Erin M.} and McConnell, {Scott R}",
year = "2019",
month = "6",
day = "1",
doi = "10.1111/jedm.12215",
language = "English (US)",
volume = "56",
pages = "437--451",
journal = "Journal of Educational Measurement",
issn = "0022-0655",
publisher = "Wiley-Blackwell",
number = "2",

}

TY - JOUR

T1 - Computerized Adaptive Testing in Early Education

T2 - Exploring the Impact of Item Position Effects on Ability Estimation

AU - Albano, Anthony D.

AU - Cai, Liuhan

AU - Lease, Erin M.

AU - McConnell, Scott R

PY - 2019/6/1

Y1 - 2019/6/1

N2 - Studies have shown that item difficulty can vary significantly based on the context of an item within a test form. In particular, item position may be associated with practice and fatigue effects that influence item parameter estimation. The purpose of this research was to examine the relevance of item position specifically for assessments used in early education, an area of testing that has received relatively limited psychometric attention. In an initial study, multilevel item response models fit to data from an early literacy measure revealed statistically significant increases in difficulty for items appearing later in a 20-item form. The estimated linear change in logits for an increase of 1 in position was.024, resulting in a predicted change of.46 logits for a shift from the beginning to the end of the form. A subsequent simulation study examined impacts of item position effects on person ability estimation within computerized adaptive testing. Implications and recommendations for practice are discussed.

AB - Studies have shown that item difficulty can vary significantly based on the context of an item within a test form. In particular, item position may be associated with practice and fatigue effects that influence item parameter estimation. The purpose of this research was to examine the relevance of item position specifically for assessments used in early education, an area of testing that has received relatively limited psychometric attention. In an initial study, multilevel item response models fit to data from an early literacy measure revealed statistically significant increases in difficulty for items appearing later in a 20-item form. The estimated linear change in logits for an increase of 1 in position was.024, resulting in a predicted change of.46 logits for a shift from the beginning to the end of the form. A subsequent simulation study examined impacts of item position effects on person ability estimation within computerized adaptive testing. Implications and recommendations for practice are discussed.

UR - http://www.scopus.com/inward/record.url?scp=85066808627&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85066808627&partnerID=8YFLogxK

U2 - 10.1111/jedm.12215

DO - 10.1111/jedm.12215

M3 - Article

AN - SCOPUS:85066808627

VL - 56

SP - 437

EP - 451

JO - Journal of Educational Measurement

JF - Journal of Educational Measurement

SN - 0022-0655

IS - 2

ER -