Assessing Item-Level Fit for Higher Order Item Response Theory Models

Xue Zhang; Chun Wang; Jian Tao

doi:10.1177/0146621618762740

Assessing Item-Level Fit for Higher Order Item Response Theory Models

Xue Zhang, Chun Wang, Jian Tao

Psychology (Twin Cities)

Research output: Contribution to journal › Article › peer-review

6 Scopus citations

Abstract

Testing item-level fit is important in scale development to guide item revision/deletion. Many item-level fit indices have been proposed in literature, yet none of them were directly applicable to an important family of models, namely, the higher order item response theory (HO-IRT) models. In this study, chi-square-based fit indices (i.e., Yen’s Q₁, McKinley and Mill’s G², Orlando and Thissen’s S-X², and S-G²) were extended to HO-IRT models. Their performances are evaluated via simulation studies in terms of false positive rates and correct detection rates. The manipulated factors include test structure (i.e., test length and number of dimensions), sample size, level of correlations among dimensions, and the proportion of misfitting items. For misfitting items, the sources of misfit, including the misfitting item response functions, and misspecifying factor structures were also manipulated. The results from simulation studies demonstrate that the S-G² is promising for higher order items.

Original language	English (US)
Pages (from-to)	644-659
Number of pages	16
Journal	Applied Psychological Measurement
Volume	42
Issue number	8
DOIs	https://doi.org/10.1177/0146621618762740
State	Published - Nov 1 2018

Bibliographical note

Funding Information:
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the National Natural Science and Social Science Foundations of China (Grant 11571069) and Institute of Education Sciences (IES) (Grant R305D160010).

Publisher Copyright:
© The Author(s) 2018.

Keywords

S-G
S-X
correct detection rate
false positive rate
higher order IRT models
item fit

Publisher link

10.1177/0146621618762740

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6291895

Find It

Find It at U of M Libraries

Cite this

@article{6746eae6e264422cb0bd94770be1b529,

title = "Assessing Item-Level Fit for Higher Order Item Response Theory Models",

abstract = "Testing item-level fit is important in scale development to guide item revision/deletion. Many item-level fit indices have been proposed in literature, yet none of them were directly applicable to an important family of models, namely, the higher order item response theory (HO-IRT) models. In this study, chi-square-based fit indices (i.e., Yen{\textquoteright}s Q1, McKinley and Mill{\textquoteright}s G2, Orlando and Thissen{\textquoteright}s S-X2, and S-G2) were extended to HO-IRT models. Their performances are evaluated via simulation studies in terms of false positive rates and correct detection rates. The manipulated factors include test structure (i.e., test length and number of dimensions), sample size, level of correlations among dimensions, and the proportion of misfitting items. For misfitting items, the sources of misfit, including the misfitting item response functions, and misspecifying factor structures were also manipulated. The results from simulation studies demonstrate that the S-G2 is promising for higher order items.",

keywords = "S-G, S-X, correct detection rate, false positive rate, higher order IRT models, item fit",

author = "Xue Zhang and Chun Wang and Jian Tao",

note = "Funding Information: The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the National Natural Science and Social Science Foundations of China (Grant 11571069) and Institute of Education Sciences (IES) (Grant R305D160010). Publisher Copyright: {\textcopyright} The Author(s) 2018.",

year = "2018",

month = nov,

day = "1",

doi = "10.1177/0146621618762740",

language = "English (US)",

volume = "42",

pages = "644--659",

journal = "Applied Psychological Measurement",

issn = "0146-6216",

publisher = "SAGE Publications Inc.",

number = "8",

}

TY - JOUR

T1 - Assessing Item-Level Fit for Higher Order Item Response Theory Models

AU - Zhang, Xue

AU - Wang, Chun

AU - Tao, Jian

N1 - Funding Information: The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the National Natural Science and Social Science Foundations of China (Grant 11571069) and Institute of Education Sciences (IES) (Grant R305D160010). Publisher Copyright: © The Author(s) 2018.

PY - 2018/11/1

Y1 - 2018/11/1

N2 - Testing item-level fit is important in scale development to guide item revision/deletion. Many item-level fit indices have been proposed in literature, yet none of them were directly applicable to an important family of models, namely, the higher order item response theory (HO-IRT) models. In this study, chi-square-based fit indices (i.e., Yen’s Q1, McKinley and Mill’s G2, Orlando and Thissen’s S-X2, and S-G2) were extended to HO-IRT models. Their performances are evaluated via simulation studies in terms of false positive rates and correct detection rates. The manipulated factors include test structure (i.e., test length and number of dimensions), sample size, level of correlations among dimensions, and the proportion of misfitting items. For misfitting items, the sources of misfit, including the misfitting item response functions, and misspecifying factor structures were also manipulated. The results from simulation studies demonstrate that the S-G2 is promising for higher order items.

AB - Testing item-level fit is important in scale development to guide item revision/deletion. Many item-level fit indices have been proposed in literature, yet none of them were directly applicable to an important family of models, namely, the higher order item response theory (HO-IRT) models. In this study, chi-square-based fit indices (i.e., Yen’s Q1, McKinley and Mill’s G2, Orlando and Thissen’s S-X2, and S-G2) were extended to HO-IRT models. Their performances are evaluated via simulation studies in terms of false positive rates and correct detection rates. The manipulated factors include test structure (i.e., test length and number of dimensions), sample size, level of correlations among dimensions, and the proportion of misfitting items. For misfitting items, the sources of misfit, including the misfitting item response functions, and misspecifying factor structures were also manipulated. The results from simulation studies demonstrate that the S-G2 is promising for higher order items.

KW - S-G

KW - S-X

KW - correct detection rate

KW - false positive rate

KW - higher order IRT models

KW - item fit

UR - http://www.scopus.com/inward/record.url?scp=85044352199&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85044352199&partnerID=8YFLogxK

U2 - 10.1177/0146621618762740

DO - 10.1177/0146621618762740

M3 - Article

C2 - 30559572

AN - SCOPUS:85044352199

SN - 0146-6216

VL - 42

SP - 644

EP - 659

JO - Applied Psychological Measurement

JF - Applied Psychological Measurement

IS - 8

ER -

Assessing Item-Level Fit for Higher Order Item Response Theory Models

Abstract

Bibliographical note

Keywords

Publisher link

Find It

Other files and links

Fingerprint

Cite this