You are what you click: using machine learning to model trace data for psychometric measurement

Richard N. Landers, Elena M Auer, Gabriel Mersy, Sebastian Marin, Jason Blaik

Research output: Contribution to journalArticlepeer-review


Assessment trace data, such as mouse positions and their timing, offer interesting and provocative reflections of individual differences yet are currently underutilized by testing professionals. In this article, we present a 10-step procedure to maximize the probability that a trace data modeling project will be successful: 1) grounding the project in psychometric theory, 2) building technical infrastructure to collect trace data, 3) designing a useful developmental validation study, 4) using a holdout validation approach with collected data, 5) using exploratory analysis to conduct meaningful feature engineering, 6) identifying useful machine learning algorithms to predict a thoughtfully chosen criterion, 7) engineering a machine learning model with meaningful internal cross-validation and hyperparameter selection, 8) conducting model diagnostics to assess if the resulting model is overfitted, underfitted, or within acceptable tolerance, and 9) testing the success of the final model in meeting conceptual, technical, and psychometric goals. If deemed successful, trace data model predictions could then be engineered into decision-making systems. We present this framework within the broader view of psychometrics, exploring the challenges of developing psychometrically valid models using such complex data with much weaker trait signals than assessment developers have typically attempted to model.

Original languageEnglish (US)
Pages (from-to)243-263
Number of pages21
JournalInternational Journal of Testing
Issue number3-4
StatePublished - 2022

Bibliographical note

Publisher Copyright:
© Copyright © 2022 The Author(s). Published with license by Taylor & Francis Group, LLC.


  • Machine learning
  • data science
  • mousetrap
  • psychometric
  • trace data


Dive into the research topics of 'You are what you click: using machine learning to model trace data for psychometric measurement'. Together they form a unique fingerprint.

Cite this