Combining CAT with cognitive diagnosis: A weighted item selection approach

Chun Wang, Hua Hua Chang, Jeffery Douglas

Research output: Contribution to journalArticlepeer-review

21 Scopus citations

Abstract

Computerized adaptive testing (CAT) was originally proposed to measure θ, usually a latent trait, with greater precision by sequentially selecting items according to the student's responses to previously administered items. Although the application of CAT is promising for many educational testing programs, most of the current CAT systems were not designed to provide diagnostic information. This article discusses item selection strategies specifically tailored for cognitive diagnostic tests. Our goal is to identify an effective item selection algorithm that not only estimates θ efficiently, but also classifies the student's knowledge status α accurately. A single-stage item selection method with a dual purpose will be introduced. The main idea is to treat diagnostic criteria as constraints: Using the maximum priority index method to meet these constraints, the CAT system is able to generate cognitive diagnostic feedback in a fairly straightforward fashion. Different priority functions are proposed. Some of them are based on certain information measures, such as Kullback-Leibler information, and others utilize only the information provided by the Q-matrix. An extensive simulation study is conducted, and the results indicate that the information-based method not only yields higher classification rates for cognitive diagnosis, but also achieves more accurate θ estimation. Other constraint controls, such as item exposure rates, are also considered for all the competing methods.

Original languageEnglish (US)
Pages (from-to)95-109
Number of pages15
JournalBehavior Research Methods
Volume44
Issue number1
DOIs
StatePublished - Mar 2012

Keywords

  • CAT
  • Cognitive diagnosis
  • Constraint-weighted item selection
  • a-stratification

Fingerprint Dive into the research topics of 'Combining CAT with cognitive diagnosis: A weighted item selection approach'. Together they form a unique fingerprint.

Cite this