A comparison of automated scale short form selection strategies

Anthony W. Raborn, Walter L. Leite, Katerina M. Marcoulides

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Short forms of psychometric scales have been commonly used in educational and psychological research to reduce the burden of test administration. However, it is challenging to select items for a short form that preserve the validity and reliability of the scores of the original scale. This paper presents and evaluates multiple automated methods for scale short form creation based on metaheuristic optimization algorithms that incorporate validity criteria based on internal structure and relationships with other variables. The ant colony optimization (ACO) algorithm, tabu search (TS), simulated annealing (SA) and genetic algorithm (GA) are examined using confirmatory factor analysis (CFA) of scales with one factor, three factor, and bi-factor factorial structure. The results indicate that SA created short forms with best model fit for scales with one and three factor structures, but ACO was able to obtain highest reliability. For scales with bi-factor structure, SA provide short forms with best model fit, but TS obtained highest reliability. Overall, the SA algorithm is recommended because it produced consistently best model fit and reliability that was only slightly lower than the ACO or TS algorithms.

Original languageEnglish (US)
Title of host publicationEDM 2019 - Proceedings of the 12th International Conference on Educational Data Mining
EditorsCollin F. Lynch, Agathe Merceron, Michel Desmarais, Roger Nkambou
PublisherInternational Educational Data Mining Society
Pages402-407
Number of pages6
ISBN (Electronic)9781733673600
StatePublished - 2019
Externally publishedYes
Event12th International Conference on Educational Data Mining, EDM 2019 - Montreal, Canada
Duration: Jul 2 2019Jul 5 2019

Publication series

NameEDM 2019 - Proceedings of the 12th International Conference on Educational Data Mining

Conference

Conference12th International Conference on Educational Data Mining, EDM 2019
CountryCanada
CityMontreal
Period7/2/197/5/19

Keywords

  • Ant colony optimization
  • Confirmatory factor analysis
  • Genetic algorithm
  • Metaheuristic algorithms
  • Short form development
  • Simulated annealing
  • Tabu search
  • Validity

Fingerprint Dive into the research topics of 'A comparison of automated scale short form selection strategies'. Together they form a unique fingerprint.

  • Cite this

    Raborn, A. W., Leite, W. L., & Marcoulides, K. M. (2019). A comparison of automated scale short form selection strategies. In C. F. Lynch, A. Merceron, M. Desmarais, & R. Nkambou (Eds.), EDM 2019 - Proceedings of the 12th International Conference on Educational Data Mining (pp. 402-407). (EDM 2019 - Proceedings of the 12th International Conference on Educational Data Mining). International Educational Data Mining Society.