A Two-Level Alternating Direction Model for Polytomous Items With Local Dependence

Igor Himelfarb, Katerina M Marcoulides, Guoliang Fang, Bruce L. Shotts

Research output: Contribution to journalArticle

Abstract

The chiropractic clinical competency examination uses groups of items that are integrated by a common case vignette. The nature of the vignette items violates the assumption of local independence for items nested within a vignette. This study examines via simulation a new algorithmic approach for addressing the local independence violation problem using a two-level alternating directions testlet model. Parameter values for item difficulty, discrimination, test-taker ability, and test-taker secondary abilities associated with a particular testlet are generated and parameter recovery through Markov Chain Monte Carlo Bayesian methods and generalized maximum likelihood estimation methods are compared. To aid with the complex computational efforts, the novel so-called TensorFlow platform is used. Both estimation methods provided satisfactory parameter recovery, although the Bayesian methods were found to be somewhat superior in recovering item discrimination parameters. The practical significance of the results are discussed in relation to obtaining accurate estimates of item, test, ability parameters, and measurement reliability information.

Original languageEnglish (US)
JournalEducational and Psychological Measurement
DOIs
StateAccepted/In press - Jan 1 2019

Fingerprint

Alternating Direction
Bayes Theorem
Chiropractic
Monte Carlo Method
Recovery
Markov Chains
Clinical Competence
Local Independence
Maximum likelihood estimation
Markov processes
Bayesian Methods
Discrimination
ability
discrimination
Model
Violate
Markov Chain Monte Carlo
Maximum Likelihood Estimation
Monte Carlo method
Direction compound

Keywords

  • Bayesian methods
  • Markov Chain Monte Carlo (MCMC)
  • generalized maximum likelihood estimation (GMLE)
  • testlet response theory (TRT)
  • violation of local independence

Cite this

A Two-Level Alternating Direction Model for Polytomous Items With Local Dependence. / Himelfarb, Igor; Marcoulides, Katerina M; Fang, Guoliang; Shotts, Bruce L.

In: Educational and Psychological Measurement, 01.01.2019.

Research output: Contribution to journalArticle

@article{4a34832f605344f7bb564e9ec0ecab15,
title = "A Two-Level Alternating Direction Model for Polytomous Items With Local Dependence",
abstract = "The chiropractic clinical competency examination uses groups of items that are integrated by a common case vignette. The nature of the vignette items violates the assumption of local independence for items nested within a vignette. This study examines via simulation a new algorithmic approach for addressing the local independence violation problem using a two-level alternating directions testlet model. Parameter values for item difficulty, discrimination, test-taker ability, and test-taker secondary abilities associated with a particular testlet are generated and parameter recovery through Markov Chain Monte Carlo Bayesian methods and generalized maximum likelihood estimation methods are compared. To aid with the complex computational efforts, the novel so-called TensorFlow platform is used. Both estimation methods provided satisfactory parameter recovery, although the Bayesian methods were found to be somewhat superior in recovering item discrimination parameters. The practical significance of the results are discussed in relation to obtaining accurate estimates of item, test, ability parameters, and measurement reliability information.",
keywords = "Bayesian methods, Markov Chain Monte Carlo (MCMC), generalized maximum likelihood estimation (GMLE), testlet response theory (TRT), violation of local independence",
author = "Igor Himelfarb and Marcoulides, {Katerina M} and Guoliang Fang and Shotts, {Bruce L.}",
year = "2019",
month = "1",
day = "1",
doi = "10.1177/0013164419871597",
language = "English (US)",
journal = "Educational and Psychological Measurement",
issn = "0013-1644",
publisher = "SAGE Publications Inc.",

}

TY - JOUR

T1 - A Two-Level Alternating Direction Model for Polytomous Items With Local Dependence

AU - Himelfarb, Igor

AU - Marcoulides, Katerina M

AU - Fang, Guoliang

AU - Shotts, Bruce L.

PY - 2019/1/1

Y1 - 2019/1/1

N2 - The chiropractic clinical competency examination uses groups of items that are integrated by a common case vignette. The nature of the vignette items violates the assumption of local independence for items nested within a vignette. This study examines via simulation a new algorithmic approach for addressing the local independence violation problem using a two-level alternating directions testlet model. Parameter values for item difficulty, discrimination, test-taker ability, and test-taker secondary abilities associated with a particular testlet are generated and parameter recovery through Markov Chain Monte Carlo Bayesian methods and generalized maximum likelihood estimation methods are compared. To aid with the complex computational efforts, the novel so-called TensorFlow platform is used. Both estimation methods provided satisfactory parameter recovery, although the Bayesian methods were found to be somewhat superior in recovering item discrimination parameters. The practical significance of the results are discussed in relation to obtaining accurate estimates of item, test, ability parameters, and measurement reliability information.

AB - The chiropractic clinical competency examination uses groups of items that are integrated by a common case vignette. The nature of the vignette items violates the assumption of local independence for items nested within a vignette. This study examines via simulation a new algorithmic approach for addressing the local independence violation problem using a two-level alternating directions testlet model. Parameter values for item difficulty, discrimination, test-taker ability, and test-taker secondary abilities associated with a particular testlet are generated and parameter recovery through Markov Chain Monte Carlo Bayesian methods and generalized maximum likelihood estimation methods are compared. To aid with the complex computational efforts, the novel so-called TensorFlow platform is used. Both estimation methods provided satisfactory parameter recovery, although the Bayesian methods were found to be somewhat superior in recovering item discrimination parameters. The practical significance of the results are discussed in relation to obtaining accurate estimates of item, test, ability parameters, and measurement reliability information.

KW - Bayesian methods

KW - Markov Chain Monte Carlo (MCMC)

KW - generalized maximum likelihood estimation (GMLE)

KW - testlet response theory (TRT)

KW - violation of local independence

UR - http://www.scopus.com/inward/record.url?scp=85072013318&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85072013318&partnerID=8YFLogxK

U2 - 10.1177/0013164419871597

DO - 10.1177/0013164419871597

M3 - Article

AN - SCOPUS:85072013318

JO - Educational and Psychological Measurement

JF - Educational and Psychological Measurement

SN - 0013-1644

ER -