(Formula presented.) is a popular item fit index that is available in commercial software packages such as flexMIRT. However, no research has systematically examined the performance of (Formula presented.) for detecting item misfit within the context of the multidimensional graded response model (MGRM). The primary goal of this study was to evaluate the performance of (Formula presented.) under two practical misfit scenarios: first, all items are misfitting due to model misspecification, and second, a small subset of items violate the underlying assumptions of the MGRM. Simulation studies showed that caution should be exercised when reporting item fit results of polytomous items using (Formula presented.) within the context of the MGRM, because of its inflated false positive rates (FPRs), especially with a small sample size and a long test. (Formula presented.) performed well when detecting overall model misfit as well as item misfit for a small subset of items when the ordinality assumption was violated. However, under a number of conditions of model misspecification or items violating the homogeneous discrimination assumption, even though true positive rates (TPRs) of (Formula presented.) were high when a small sample size was coupled with a long test, the inflated FPRs were generally directly related to increasing TPRs. There was also a suggestion that performance of (Formula presented.) was affected by the magnitude of misfit within an item. There was no evidence that FPRs for fitting items were exacerbated by the presence of a small percentage of misfitting items among them.
Bibliographical noteFunding Information:
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the Eunice Kennedy Shriver National Institutes of Child Health and Human Development of the National Institutes of Health under Award Number R01HD079439 to the Mayo Clinic in Rochester Minnesota through a subcontract to the University of Minnesota. None of the funders or sponsors of this research had any role in the design and conduct of the study; collection, management, analysis, and interpretation of data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.
- item fit
- item response theory (IRT)
- multidimensional graded response model (MGRM)