Using machine learning to model trace behavioral data from a game-based assessment

Elena M Auer, Gabriel Mersy, Sebastian Marin, Jason Blaik, Richard N. Landers

Research output: Contribution to journalArticlepeer-review

1 Scopus citations

Abstract

In the context of game-based assessments (GBAs), we examined the potential of trace data modeling to supplement or replace an existing GBA's scoring approach. We used data science tools and “big data” practices, such as feature engineering and a series of machine learning algorithms, to predict traditionally measured cognitive ability and conscientiousness scores from the copious trace data generated by a theory-driven GBA designed to measure cognitive ability. Several types of predictors were developed from the raw trace data from 621 participants, including counts of game objects that the player interacted with, the amount of time spent doing so, and mouse movement data across a variety of meaningful intervals. Broadly, we found promising evidence for trace data modeling of cognitive ability, including incremental contribution to the prediction of a criterion grade point average (GPA), but less promising evidence for trace data modeling of conscientiousness, suggesting that trace data modeling like this may be more valuable for assessing traits in games that were developed to target those traits.

Original languageEnglish (US)
Pages (from-to)82-102
Number of pages21
JournalInternational Journal of Selection and Assessment
Volume30
Issue number1
DOIs
StatePublished - Mar 2022

Bibliographical note

Funding Information:
Participant payments in this study were funded by Revelian Pty Ltd.

Publisher Copyright:
© 2021 John Wiley & Sons Ltd.

Keywords

  • data science
  • game-based assessments
  • games
  • machine learning
  • microbehaviors
  • psychometrics
  • trace data

Fingerprint

Dive into the research topics of 'Using machine learning to model trace behavioral data from a game-based assessment'. Together they form a unique fingerprint.

Cite this