Skip to main navigation Skip to search Skip to main content

What do assessment center ratings reflect? Consistency and heterogeneity in variance composition across multiple samples

  • Pia V. Ingold
  • , Anna Luca Heimann
  • , Bettina Waller
  • , Simon M. Breil
  • , Paul R. Sackett

Research output: Contribution to journal › Article › peer-review

Abstract

The question of what assessment centers' measure has remained a controversial topic in research for decades, with a recent increase in studies that (a) use generalizability theory and (b) acknowledge the effects of aggregating postexercise dimension ratings into higher level assessment center scores. Building on these developments, we used Bayesian generalizability theory and random-effects meta-analyses to examine the variance explained by assessment center components such as assessees, exercises, dimensions, assessors, their interactions, and the interrater reliability of AC ratings in 19 different assessment center samples from various organizations (N = 4,963 assessees with 272,528 observations). This provides the first meta-analytic estimates of these effects, as well as insight into the extent to which findings from previous studies generalize to assessment center samples that differ in measurement design, industry, and purpose, and how heterogeneous these effects are across samples. Results were consistent with previous trends in the ranking of variance explained by key AC components (with assessee main effects and assessee-exercise effects being the largest variance components) and additionally emphasized the relevance of assessee-exercise-dimension effects. In addition, meta-analytic results suggested substantial heterogeneity in all reliable variance components (i.e., assessee main effect, assessee-exercise effect, assessee-dimension effect, and assessee-exercise-dimension effect) and in interrater reliability across assessment center samples. Aggregating AC ratings into higher level scores (i.e., overall AC scores, exercise-level scores, and dimension-level scores) reduced heterogeneity only slightly. Implications of the findings for a multifaceted assessment center functioning are discussed. (PsycInfo Database Record (c) 2026 APA, all rights reserved).

Original languageEnglish (US)
Pages (from-to)324-346
Number of pages23
JournalThe Journal of applied psychology
Volume111
Issue number3
DOIs
StatePublished - Mar 1 2026

PubMed: MeSH publication types

  • Journal Article
  • Meta-Analysis

Fingerprint

Dive into the research topics of 'What do assessment center ratings reflect? Consistency and heterogeneity in variance composition across multiple samples'. Together they form a unique fingerprint.

Cite this