Multivariate Hypothesis Testing Methods for Evaluating Significant Individual Change

Chun Wang, David J. Weiss

Research output: Contribution to journalArticlepeer-review

6 Scopus citations

Abstract

The measurement of individual change has been an important topic in both education and psychology. For instance, teachers are interested in whether students have significantly improved (e.g., learned) from instruction, and counselors are interested in whether particular behaviors have been significantly changed after certain interventions. Although classical test methods have been unable to adequately resolve the problems in measuring change, recent approaches for measuring change have begun to use item response theory (IRT). However, all prior methods mainly focus on testing whether growth is significant at the group level. The present research targets a key research question: Is the “change” in latent trait estimates for each individual significant across occasions? Many researchers have addressed this research question assuming that the latent trait is unidimensional. This research generalizes their earlier work and proposes four hypothesis testing methods to evaluate individual change on multiple latent traits: a multivariate Z-test, a multivariate likelihood ratio test, a multivariate score test, and a Kullback–Leibler test. Simulation results show that these tests hold promise of detecting individual change with low Type I error and high power. A real-data example from an educational assessment illustrates the application of the proposed methods.

Original languageEnglish (US)
Pages (from-to)221-239
Number of pages19
JournalApplied Psychological Measurement
Volume42
Issue number3
DOIs
StatePublished - May 1 2018

Bibliographical note

Funding Information:
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This article is partially supported by Institute of Education Sciences (IES) Grant R305D160010, National Institutes of Health (NIH) Grant R01HD079439-01A1, and Spencer Foundation.

Publisher Copyright:
© 2017, © The Author(s) 2017.

Keywords

  • Kullback–Leibler test
  • individual change
  • likelihood ratio test
  • multidimensional item response theory
  • score test

Fingerprint

Dive into the research topics of 'Multivariate Hypothesis Testing Methods for Evaluating Significant Individual Change'. Together they form a unique fingerprint.

Cite this