Abstract
Most computerized adaptive tests (CATs) have been studied using the framework of unidimensional item response theory. However, many psychological variables are multidimensional and might benefit from using a multidimensional approach to CATs. This study investigated the accuracy, fidelity, and efficiency of a fully multidimensional CAT algorithm (MCAT) with a bifactor model using simulated data. Four item selection methods in MCAT were examined for three bifactor pattern designs using two multidimensional item response theory models. To compare MCAT item selection and estimation methods, a fixed test length was used. The Ds-optimality item selection improved θ estimates with respect to a general factor, and either D- or A-optimality improved estimates of the group factors in three bifactor pattern designs under two multidimensional item response theory models. The MCAT model without a guessing parameter functioned better than the MCAT model with a guessing parameter. The MAP (maximum a posteriori) estimation method provided more accurate θ estimates than the EAP (expected a posteriori) method under most conditions, and MAP showed lower observed standard errors than EAP under most conditions, except for a general factor condition using Ds-optimality item selection.
| Original language | English (US) |
|---|---|
| Pages (from-to) | 954-978 |
| Number of pages | 25 |
| Journal | Educational and Psychological Measurement |
| Volume | 75 |
| Issue number | 6 |
| DOIs | |
| State | Published - Dec 1 2015 |
Fingerprint
Keywords
- bifactor model
- computerized adaptive testing
- full information item factor analysis
- multidimensional item response theory
Cite this
Best Design for Multidimensional Computerized Adaptive Testing With the Bifactor Model. / Seo, Dong Gi; Weiss, David J.
In: Educational and Psychological Measurement, Vol. 75, No. 6, 01.12.2015, p. 954-978.Research output: Contribution to journal › Article
}
TY - JOUR
T1 - Best Design for Multidimensional Computerized Adaptive Testing With the Bifactor Model
AU - Seo, Dong Gi
AU - Weiss, David J
PY - 2015/12/1
Y1 - 2015/12/1
N2 - Most computerized adaptive tests (CATs) have been studied using the framework of unidimensional item response theory. However, many psychological variables are multidimensional and might benefit from using a multidimensional approach to CATs. This study investigated the accuracy, fidelity, and efficiency of a fully multidimensional CAT algorithm (MCAT) with a bifactor model using simulated data. Four item selection methods in MCAT were examined for three bifactor pattern designs using two multidimensional item response theory models. To compare MCAT item selection and estimation methods, a fixed test length was used. The Ds-optimality item selection improved θ estimates with respect to a general factor, and either D- or A-optimality improved estimates of the group factors in three bifactor pattern designs under two multidimensional item response theory models. The MCAT model without a guessing parameter functioned better than the MCAT model with a guessing parameter. The MAP (maximum a posteriori) estimation method provided more accurate θ estimates than the EAP (expected a posteriori) method under most conditions, and MAP showed lower observed standard errors than EAP under most conditions, except for a general factor condition using Ds-optimality item selection.
AB - Most computerized adaptive tests (CATs) have been studied using the framework of unidimensional item response theory. However, many psychological variables are multidimensional and might benefit from using a multidimensional approach to CATs. This study investigated the accuracy, fidelity, and efficiency of a fully multidimensional CAT algorithm (MCAT) with a bifactor model using simulated data. Four item selection methods in MCAT were examined for three bifactor pattern designs using two multidimensional item response theory models. To compare MCAT item selection and estimation methods, a fixed test length was used. The Ds-optimality item selection improved θ estimates with respect to a general factor, and either D- or A-optimality improved estimates of the group factors in three bifactor pattern designs under two multidimensional item response theory models. The MCAT model without a guessing parameter functioned better than the MCAT model with a guessing parameter. The MAP (maximum a posteriori) estimation method provided more accurate θ estimates than the EAP (expected a posteriori) method under most conditions, and MAP showed lower observed standard errors than EAP under most conditions, except for a general factor condition using Ds-optimality item selection.
KW - bifactor model
KW - computerized adaptive testing
KW - full information item factor analysis
KW - multidimensional item response theory
UR - http://www.scopus.com/inward/record.url?scp=84947460548&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84947460548&partnerID=8YFLogxK
U2 - 10.1177/0013164415575147
DO - 10.1177/0013164415575147
M3 - Article
VL - 75
SP - 954
EP - 978
JO - Educational and Psychological Measurement
JF - Educational and Psychological Measurement
SN - 0013-1644
IS - 6
ER -