A simulation of the impacts of machine learning to combine psychometric employee selection system predictors on performance prediction, adverse impact, and number of dropped predictors

Richard N. Landers, Elena M Auer, Lily Dunk, Markus Langer, Khue Tran

Research output: Contribution to journalArticlepeer-review

3 Scopus citations

Abstract

We compare modern machine learning (MML) techniques to ordinary least squares (OLS) regression on out-of-sample (OOS) operational validity, adverse impact, and dropped predictor counts within a common selection scenario: the prediction of job performance from a battery of diverse psychometrically-validated tests. In total, scores from 1.2 billion validation study participants were simulated to describe outcomes across 31,752 combinations selection system design and scoring decisions. The most consistently valuable improvement from adopting MML over traditional regression was from dropping predictors rather than by improving prediction. On average, MML improved prediction of performance from psychometric scale composites only when the ratio of sample size to scale count was less than approximately 3, although algorithm choice, predictor count, and selection ratio affected outcomes as well. We also simulated the effects of design choices when combining item scores, which showed consistent, superior predictive accuracy for several MML algorithms, especially elastic net and random forest, over traditional regression. Given these results, we suggest the potential of machine learning for employee selection is unlikely to be realized in selection systems focusing on the combination of scale composites from previously validated psychometric tests. Instead, it will be realized in unconventional design scenarios, such as the use of individual items to make multiple trait inferences, or with novel data formats like text, image, audio, video, and behavioral traces. We therefore recommend researchers focus on the potential value of MML in future selection contexts rather than continuing to focus on the current value of MML in current selection contexts.

Original languageEnglish (US)
Pages (from-to)1037-1060
Number of pages24
JournalPersonnel Psychology
Volume76
Issue number4
DOIs
StatePublished - Dec 1 2023

Bibliographical note

Publisher Copyright:
© 2023 The Authors. Personnel Psychology published by Wiley Periodicals LLC.

Keywords

  • employee selection
  • machine learning
  • simulation

Fingerprint

Dive into the research topics of 'A simulation of the impacts of machine learning to combine psychometric employee selection system predictors on performance prediction, adverse impact, and number of dropped predictors'. Together they form a unique fingerprint.

Cite this