Best Design for Multidimensional Computerized Adaptive Testing With the Bifactor Model

Dong Gi Seo, David J Weiss

Research output: Contribution to journalArticle

6 Citations (Scopus)

Abstract

Most computerized adaptive tests (CATs) have been studied using the framework of unidimensional item response theory. However, many psychological variables are multidimensional and might benefit from using a multidimensional approach to CATs. This study investigated the accuracy, fidelity, and efficiency of a fully multidimensional CAT algorithm (MCAT) with a bifactor model using simulated data. Four item selection methods in MCAT were examined for three bifactor pattern designs using two multidimensional item response theory models. To compare MCAT item selection and estimation methods, a fixed test length was used. The Ds-optimality item selection improved θ estimates with respect to a general factor, and either D- or A-optimality improved estimates of the group factors in three bifactor pattern designs under two multidimensional item response theory models. The MCAT model without a guessing parameter functioned better than the MCAT model with a guessing parameter. The MAP (maximum a posteriori) estimation method provided more accurate θ estimates than the EAP (expected a posteriori) method under most conditions, and MAP showed lower observed standard errors than EAP under most conditions, except for a general factor condition using Ds-optimality item selection.

Original languageEnglish (US)
Pages (from-to)954-978
Number of pages25
JournalEducational and Psychological Measurement
Volume75
Issue number6
DOIs
StatePublished - Dec 1 2015

Fingerprint

Adaptive Testing
CAT
Testing
model theory
Adaptive Test
Design Patterns
Model
Model Theory
Optimality
A-optimality
Estimate
Maximum a Posteriori Estimation
Design
Psychology
Maximum a Posteriori
efficiency
Standard error
Fidelity
Group

Keywords

  • bifactor model
  • computerized adaptive testing
  • full information item factor analysis
  • multidimensional item response theory

Cite this

Best Design for Multidimensional Computerized Adaptive Testing With the Bifactor Model. / Seo, Dong Gi; Weiss, David J.

In: Educational and Psychological Measurement, Vol. 75, No. 6, 01.12.2015, p. 954-978.

Research output: Contribution to journalArticle

@article{5ab1271e4f204d4a93a650c803966b08,
title = "Best Design for Multidimensional Computerized Adaptive Testing With the Bifactor Model",
abstract = "Most computerized adaptive tests (CATs) have been studied using the framework of unidimensional item response theory. However, many psychological variables are multidimensional and might benefit from using a multidimensional approach to CATs. This study investigated the accuracy, fidelity, and efficiency of a fully multidimensional CAT algorithm (MCAT) with a bifactor model using simulated data. Four item selection methods in MCAT were examined for three bifactor pattern designs using two multidimensional item response theory models. To compare MCAT item selection and estimation methods, a fixed test length was used. The Ds-optimality item selection improved θ estimates with respect to a general factor, and either D- or A-optimality improved estimates of the group factors in three bifactor pattern designs under two multidimensional item response theory models. The MCAT model without a guessing parameter functioned better than the MCAT model with a guessing parameter. The MAP (maximum a posteriori) estimation method provided more accurate θ estimates than the EAP (expected a posteriori) method under most conditions, and MAP showed lower observed standard errors than EAP under most conditions, except for a general factor condition using Ds-optimality item selection.",
keywords = "bifactor model, computerized adaptive testing, full information item factor analysis, multidimensional item response theory",
author = "Seo, {Dong Gi} and Weiss, {David J}",
year = "2015",
month = "12",
day = "1",
doi = "10.1177/0013164415575147",
language = "English (US)",
volume = "75",
pages = "954--978",
journal = "Educational and Psychological Measurement",
issn = "0013-1644",
publisher = "SAGE Publications Inc.",
number = "6",

}

TY - JOUR

T1 - Best Design for Multidimensional Computerized Adaptive Testing With the Bifactor Model

AU - Seo, Dong Gi

AU - Weiss, David J

PY - 2015/12/1

Y1 - 2015/12/1

N2 - Most computerized adaptive tests (CATs) have been studied using the framework of unidimensional item response theory. However, many psychological variables are multidimensional and might benefit from using a multidimensional approach to CATs. This study investigated the accuracy, fidelity, and efficiency of a fully multidimensional CAT algorithm (MCAT) with a bifactor model using simulated data. Four item selection methods in MCAT were examined for three bifactor pattern designs using two multidimensional item response theory models. To compare MCAT item selection and estimation methods, a fixed test length was used. The Ds-optimality item selection improved θ estimates with respect to a general factor, and either D- or A-optimality improved estimates of the group factors in three bifactor pattern designs under two multidimensional item response theory models. The MCAT model without a guessing parameter functioned better than the MCAT model with a guessing parameter. The MAP (maximum a posteriori) estimation method provided more accurate θ estimates than the EAP (expected a posteriori) method under most conditions, and MAP showed lower observed standard errors than EAP under most conditions, except for a general factor condition using Ds-optimality item selection.

AB - Most computerized adaptive tests (CATs) have been studied using the framework of unidimensional item response theory. However, many psychological variables are multidimensional and might benefit from using a multidimensional approach to CATs. This study investigated the accuracy, fidelity, and efficiency of a fully multidimensional CAT algorithm (MCAT) with a bifactor model using simulated data. Four item selection methods in MCAT were examined for three bifactor pattern designs using two multidimensional item response theory models. To compare MCAT item selection and estimation methods, a fixed test length was used. The Ds-optimality item selection improved θ estimates with respect to a general factor, and either D- or A-optimality improved estimates of the group factors in three bifactor pattern designs under two multidimensional item response theory models. The MCAT model without a guessing parameter functioned better than the MCAT model with a guessing parameter. The MAP (maximum a posteriori) estimation method provided more accurate θ estimates than the EAP (expected a posteriori) method under most conditions, and MAP showed lower observed standard errors than EAP under most conditions, except for a general factor condition using Ds-optimality item selection.

KW - bifactor model

KW - computerized adaptive testing

KW - full information item factor analysis

KW - multidimensional item response theory

UR - http://www.scopus.com/inward/record.url?scp=84947460548&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84947460548&partnerID=8YFLogxK

U2 - 10.1177/0013164415575147

DO - 10.1177/0013164415575147

M3 - Article

VL - 75

SP - 954

EP - 978

JO - Educational and Psychological Measurement

JF - Educational and Psychological Measurement

SN - 0013-1644

IS - 6

ER -