Abstract
Expert judges often claim to utilize expert insight to tailor judgments to maximize predictive validity for a specific context. We evaluated multi-organizational assessment data regarding the prediction of supervisory ratings of job performance from ratings on individual assessment dimensions, finding no evidence that the average expert assessor effectively tailored judgments to specific organizations to maximize prediction. Expert judgment was outperformed in all organizational contexts by linear models of expert judgment, optimal weighted regression models, as well as simple sum composites. Critically, the dimension weighting policies of the expert assessors were not consistent with optimal weights for predicting job performance at any organization. We discuss why expertise tends not to contribute to predictive validity and describe methods for improving overall judgmental accuracy.
| Original language | English (US) |
|---|---|
| Pages (from-to) | 202-215 |
| Number of pages | 14 |
| Journal | International Journal of Selection and Assessment |
| Volume | 30 |
| Issue number | 2 |
| DOIs | |
| State | Published - Jun 2022 |
Bibliographical note
Publisher Copyright:© 2021 John Wiley & Sons Ltd.
Keywords
- assessment
- decision making
- expert judgment
- judgment
- mechanical
- prediction