Comparing Two Algorithms for Calibrating the Restricted Non-Compensatory Multidimensional IRT Model

Chun Wang, Steven W. Nydick

Research output: Contribution to journalArticlepeer-review

33 Scopus citations

Abstract

The non-compensatory class of multidimensional item response theory (MIRT) models frequently represents the cognitive processes underlying a series of test items better than the compensatory class of MIRT models. Nevertheless, few researchers have used non-compensatory MIRT in modeling psychological data. One reason for this lack of use is because non-compensatory MIRT item parameters are notoriously difficult to accurately estimate. In this article, we propose methods to improve the estimability of a specific non-compensatory model. To initiate the discussion, we address the non-identifiability of the explored non-compensatory MIRT model by suggesting that practitioners use an item-dimension constraint matrix (namely, a Q-matrix) that results in model identifiability. We then compare two promising algorithms for high-dimensional model calibration, Markov chain Monte Carlo (MCMC) and Metropolis–Hastings Robbins–Monro (MH-RM), and discuss, via analytical demonstrations, the challenges in estimating model parameters. Based on simulation studies, we show that when the dimensions are not highly correlated, and when the Q-matrix displays appropriate structure, the non-compensatory MIRT model can be accurately calibrated (using the aforementioned methods) with as few as 1,000 people. Based on the simulations, we conclude that the MCMC algorithm is better able to estimate model parameters across a variety of conditions, whereas the MH-RM algorithm should be used with caution when a test displays complex structure and when the latent dimensions are highly correlated.

Original languageEnglish (US)
Pages (from-to)119-134
Number of pages16
JournalApplied Psychological Measurement
Volume39
Issue number2
DOIs
StatePublished - Mar 14 2015

Bibliographical note

Publisher Copyright:
© The Author(s) 2014.

Keywords

  • MCMC
  • Metropolis–Hastings Robbins–Monro
  • multidimensional IRT

Fingerprint

Dive into the research topics of 'Comparing Two Algorithms for Calibrating the Restricted Non-Compensatory Multidimensional IRT Model'. Together they form a unique fingerprint.

Cite this