Robustness of Parameter Estimation to Assumptions of Normality in the Multidimensional Graded Response Model

Chun Wang, Shiyang Su, David J Weiss

Research output: Contribution to journalArticle

6 Citations (Scopus)

Abstract

A central assumption that is implicit in estimating item parameters in item response theory (IRT) models is the normality of the latent trait distribution, whereas a similar assumption made in categorical confirmatory factor analysis (CCFA) models is the multivariate normality of the latent response variables. Violation of the normality assumption can lead to biased parameter estimates. Although previous studies have focused primarily on unidimensional IRT models, this study extended the literature by considering a multidimensional IRT model for polytomous responses, namely the multidimensional graded response model. Moreover, this study is one of few studies that specifically compared the performance of full-information maximum likelihood (FIML) estimation versus robust weighted least squares (WLS) estimation when the normality assumption is violated. The research also manipulated the number of nonnormal latent trait dimensions. Results showed that FIML consistently outperformed WLS when there were one or multiple skewed latent trait distributions. More interestingly, the bias of the discrimination parameters was non-ignorable only when the corresponding factor was skewed. Having other skewed factors did not further exacerbate the bias, whereas biases of boundary parameters increased as more nonnormal factors were added. The item parameter standard errors recovered well with both estimation algorithms regardless of the number of nonnormal dimensions.

Original languageEnglish (US)
Pages (from-to)403-418
Number of pages16
JournalMultivariate Behavioral Research
Volume53
Issue number3
DOIs
StatePublished - May 4 2018

Fingerprint

Least-Squares Analysis
Normality
Parameter Estimation
Latent Trait
Robustness
Model Theory
Statistical Factor Analysis
Weighted Least Squares
Research
Model
Multivariate Normality
Confirmatory Factor Analysis
Least Squares Estimation
Standard error
Estimation Algorithms
Maximum Likelihood Estimation
Categorical
Discrimination
Biased
Maximum Likelihood

Keywords

  • IRT
  • Ordinal data analysis
  • item response theory
  • maximum likelihood estimation
  • weighted least squares estimation

PubMed: MeSH publication types

  • Journal Article

Cite this

Robustness of Parameter Estimation to Assumptions of Normality in the Multidimensional Graded Response Model. / Wang, Chun; Su, Shiyang; Weiss, David J.

In: Multivariate Behavioral Research, Vol. 53, No. 3, 04.05.2018, p. 403-418.

Research output: Contribution to journalArticle

@article{9637cbdafbbd4850b954ee85558a3dd1,
title = "Robustness of Parameter Estimation to Assumptions of Normality in the Multidimensional Graded Response Model",
abstract = "A central assumption that is implicit in estimating item parameters in item response theory (IRT) models is the normality of the latent trait distribution, whereas a similar assumption made in categorical confirmatory factor analysis (CCFA) models is the multivariate normality of the latent response variables. Violation of the normality assumption can lead to biased parameter estimates. Although previous studies have focused primarily on unidimensional IRT models, this study extended the literature by considering a multidimensional IRT model for polytomous responses, namely the multidimensional graded response model. Moreover, this study is one of few studies that specifically compared the performance of full-information maximum likelihood (FIML) estimation versus robust weighted least squares (WLS) estimation when the normality assumption is violated. The research also manipulated the number of nonnormal latent trait dimensions. Results showed that FIML consistently outperformed WLS when there were one or multiple skewed latent trait distributions. More interestingly, the bias of the discrimination parameters was non-ignorable only when the corresponding factor was skewed. Having other skewed factors did not further exacerbate the bias, whereas biases of boundary parameters increased as more nonnormal factors were added. The item parameter standard errors recovered well with both estimation algorithms regardless of the number of nonnormal dimensions.",
keywords = "IRT, Ordinal data analysis, item response theory, maximum likelihood estimation, weighted least squares estimation",
author = "Chun Wang and Shiyang Su and Weiss, {David J}",
year = "2018",
month = "5",
day = "4",
doi = "10.1080/00273171.2018.1455572",
language = "English (US)",
volume = "53",
pages = "403--418",
journal = "Multivariate Behavioral Research",
issn = "0027-3171",
publisher = "Psychology Press Ltd",
number = "3",

}

TY - JOUR

T1 - Robustness of Parameter Estimation to Assumptions of Normality in the Multidimensional Graded Response Model

AU - Wang, Chun

AU - Su, Shiyang

AU - Weiss, David J

PY - 2018/5/4

Y1 - 2018/5/4

N2 - A central assumption that is implicit in estimating item parameters in item response theory (IRT) models is the normality of the latent trait distribution, whereas a similar assumption made in categorical confirmatory factor analysis (CCFA) models is the multivariate normality of the latent response variables. Violation of the normality assumption can lead to biased parameter estimates. Although previous studies have focused primarily on unidimensional IRT models, this study extended the literature by considering a multidimensional IRT model for polytomous responses, namely the multidimensional graded response model. Moreover, this study is one of few studies that specifically compared the performance of full-information maximum likelihood (FIML) estimation versus robust weighted least squares (WLS) estimation when the normality assumption is violated. The research also manipulated the number of nonnormal latent trait dimensions. Results showed that FIML consistently outperformed WLS when there were one or multiple skewed latent trait distributions. More interestingly, the bias of the discrimination parameters was non-ignorable only when the corresponding factor was skewed. Having other skewed factors did not further exacerbate the bias, whereas biases of boundary parameters increased as more nonnormal factors were added. The item parameter standard errors recovered well with both estimation algorithms regardless of the number of nonnormal dimensions.

AB - A central assumption that is implicit in estimating item parameters in item response theory (IRT) models is the normality of the latent trait distribution, whereas a similar assumption made in categorical confirmatory factor analysis (CCFA) models is the multivariate normality of the latent response variables. Violation of the normality assumption can lead to biased parameter estimates. Although previous studies have focused primarily on unidimensional IRT models, this study extended the literature by considering a multidimensional IRT model for polytomous responses, namely the multidimensional graded response model. Moreover, this study is one of few studies that specifically compared the performance of full-information maximum likelihood (FIML) estimation versus robust weighted least squares (WLS) estimation when the normality assumption is violated. The research also manipulated the number of nonnormal latent trait dimensions. Results showed that FIML consistently outperformed WLS when there were one or multiple skewed latent trait distributions. More interestingly, the bias of the discrimination parameters was non-ignorable only when the corresponding factor was skewed. Having other skewed factors did not further exacerbate the bias, whereas biases of boundary parameters increased as more nonnormal factors were added. The item parameter standard errors recovered well with both estimation algorithms regardless of the number of nonnormal dimensions.

KW - IRT

KW - Ordinal data analysis

KW - item response theory

KW - maximum likelihood estimation

KW - weighted least squares estimation

UR - http://www.scopus.com/inward/record.url?scp=85045035928&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85045035928&partnerID=8YFLogxK

U2 - 10.1080/00273171.2018.1455572

DO - 10.1080/00273171.2018.1455572

M3 - Article

VL - 53

SP - 403

EP - 418

JO - Multivariate Behavioral Research

JF - Multivariate Behavioral Research

SN - 0027-3171

IS - 3

ER -