TY - JOUR
T1 - Combining CAT with cognitive diagnosis
T2 - A weighted item selection approach
AU - Wang, Chun
AU - Chang, Hua Hua
AU - Douglas, Jeffery
N1 - Copyright:
Copyright 2012 Elsevier B.V., All rights reserved.
PY - 2012/3
Y1 - 2012/3
N2 - Computerized adaptive testing (CAT) was originally proposed to measure θ, usually a latent trait, with greater precision by sequentially selecting items according to the student's responses to previously administered items. Although the application of CAT is promising for many educational testing programs, most of the current CAT systems were not designed to provide diagnostic information. This article discusses item selection strategies specifically tailored for cognitive diagnostic tests. Our goal is to identify an effective item selection algorithm that not only estimates θ efficiently, but also classifies the student's knowledge status α accurately. A single-stage item selection method with a dual purpose will be introduced. The main idea is to treat diagnostic criteria as constraints: Using the maximum priority index method to meet these constraints, the CAT system is able to generate cognitive diagnostic feedback in a fairly straightforward fashion. Different priority functions are proposed. Some of them are based on certain information measures, such as Kullback-Leibler information, and others utilize only the information provided by the Q-matrix. An extensive simulation study is conducted, and the results indicate that the information-based method not only yields higher classification rates for cognitive diagnosis, but also achieves more accurate θ estimation. Other constraint controls, such as item exposure rates, are also considered for all the competing methods.
AB - Computerized adaptive testing (CAT) was originally proposed to measure θ, usually a latent trait, with greater precision by sequentially selecting items according to the student's responses to previously administered items. Although the application of CAT is promising for many educational testing programs, most of the current CAT systems were not designed to provide diagnostic information. This article discusses item selection strategies specifically tailored for cognitive diagnostic tests. Our goal is to identify an effective item selection algorithm that not only estimates θ efficiently, but also classifies the student's knowledge status α accurately. A single-stage item selection method with a dual purpose will be introduced. The main idea is to treat diagnostic criteria as constraints: Using the maximum priority index method to meet these constraints, the CAT system is able to generate cognitive diagnostic feedback in a fairly straightforward fashion. Different priority functions are proposed. Some of them are based on certain information measures, such as Kullback-Leibler information, and others utilize only the information provided by the Q-matrix. An extensive simulation study is conducted, and the results indicate that the information-based method not only yields higher classification rates for cognitive diagnosis, but also achieves more accurate θ estimation. Other constraint controls, such as item exposure rates, are also considered for all the competing methods.
KW - CAT
KW - Cognitive diagnosis
KW - Constraint-weighted item selection
KW - a-stratification
UR - http://www.scopus.com/inward/record.url?scp=84863147461&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84863147461&partnerID=8YFLogxK
U2 - 10.3758/s13428-011-0143-3
DO - 10.3758/s13428-011-0143-3
M3 - Article
C2 - 21853408
AN - SCOPUS:84863147461
SN - 1554-351X
VL - 44
SP - 95
EP - 109
JO - Behavior Research Methods
JF - Behavior Research Methods
IS - 1
ER -