Robustness, Sensitivity, and Sampling Variability of Pareto-Optimal Selection System Solutions to Address the Quality-Diversity Trade-Off

Wilfried De Corte, Paul R. Sackett, Filip Lievens

Research output: Contribution to journalArticle

Abstract

In case that both the goals of selection quality and diversity are important, a selection system is Pareto-optimal (PO) when its implementation is expected to result in an optimal balance between the levels achieved with respect to both these goals. The study addresses the critical issue whether PO systems, as computed from calibration conditions, continue to perform well when applied to a large variety of different validation selection situations. To address the key issue, we introduce two new measures for gauging the achievement of these designs and conduct a large simulation study in which we manipulate 10 factors (related to the selection situation, sensitivity/robustness, and the selection system) that cumulate in a design with 3,888 cells and 24 selection systems. Results demonstrate that PO systems are superior to other, non-PO systems (including unit weighed system designs) both in terms of the achievement measures as well as in terms of yielding more often a better quality/diversity trade-off. The study also identifies a number of conditions that favor the achievement of PO systems in realistic selection situations.

Original languageEnglish (US)
JournalOrganizational Research Methods
DOIs
StatePublished - Jan 1 2019

Fingerprint

Optimal systems
Sampling
Gaging
Systems analysis
Calibration
Trade-offs
Selection system
Robustness

Keywords

  • Pareto-optimal
  • adverse impact
  • personnel selection
  • robustness
  • sampling variability
  • selection design
  • sensitivity

Cite this

@article{7e27c13d9a9044a3a6e102eabdadecaf,
title = "Robustness, Sensitivity, and Sampling Variability of Pareto-Optimal Selection System Solutions to Address the Quality-Diversity Trade-Off",
abstract = "In case that both the goals of selection quality and diversity are important, a selection system is Pareto-optimal (PO) when its implementation is expected to result in an optimal balance between the levels achieved with respect to both these goals. The study addresses the critical issue whether PO systems, as computed from calibration conditions, continue to perform well when applied to a large variety of different validation selection situations. To address the key issue, we introduce two new measures for gauging the achievement of these designs and conduct a large simulation study in which we manipulate 10 factors (related to the selection situation, sensitivity/robustness, and the selection system) that cumulate in a design with 3,888 cells and 24 selection systems. Results demonstrate that PO systems are superior to other, non-PO systems (including unit weighed system designs) both in terms of the achievement measures as well as in terms of yielding more often a better quality/diversity trade-off. The study also identifies a number of conditions that favor the achievement of PO systems in realistic selection situations.",
keywords = "Pareto-optimal, adverse impact, personnel selection, robustness, sampling variability, selection design, sensitivity",
author = "{De Corte}, Wilfried and Sackett, {Paul R.} and Filip Lievens",
year = "2019",
month = "1",
day = "1",
doi = "10.1177/1094428118825301",
language = "English (US)",
journal = "Organizational Research Methods",
issn = "1094-4281",
publisher = "SAGE Publications Inc.",

}

TY - JOUR

T1 - Robustness, Sensitivity, and Sampling Variability of Pareto-Optimal Selection System Solutions to Address the Quality-Diversity Trade-Off

AU - De Corte, Wilfried

AU - Sackett, Paul R.

AU - Lievens, Filip

PY - 2019/1/1

Y1 - 2019/1/1

N2 - In case that both the goals of selection quality and diversity are important, a selection system is Pareto-optimal (PO) when its implementation is expected to result in an optimal balance between the levels achieved with respect to both these goals. The study addresses the critical issue whether PO systems, as computed from calibration conditions, continue to perform well when applied to a large variety of different validation selection situations. To address the key issue, we introduce two new measures for gauging the achievement of these designs and conduct a large simulation study in which we manipulate 10 factors (related to the selection situation, sensitivity/robustness, and the selection system) that cumulate in a design with 3,888 cells and 24 selection systems. Results demonstrate that PO systems are superior to other, non-PO systems (including unit weighed system designs) both in terms of the achievement measures as well as in terms of yielding more often a better quality/diversity trade-off. The study also identifies a number of conditions that favor the achievement of PO systems in realistic selection situations.

AB - In case that both the goals of selection quality and diversity are important, a selection system is Pareto-optimal (PO) when its implementation is expected to result in an optimal balance between the levels achieved with respect to both these goals. The study addresses the critical issue whether PO systems, as computed from calibration conditions, continue to perform well when applied to a large variety of different validation selection situations. To address the key issue, we introduce two new measures for gauging the achievement of these designs and conduct a large simulation study in which we manipulate 10 factors (related to the selection situation, sensitivity/robustness, and the selection system) that cumulate in a design with 3,888 cells and 24 selection systems. Results demonstrate that PO systems are superior to other, non-PO systems (including unit weighed system designs) both in terms of the achievement measures as well as in terms of yielding more often a better quality/diversity trade-off. The study also identifies a number of conditions that favor the achievement of PO systems in realistic selection situations.

KW - Pareto-optimal

KW - adverse impact

KW - personnel selection

KW - robustness

KW - sampling variability

KW - selection design

KW - sensitivity

UR - http://www.scopus.com/inward/record.url?scp=85061176385&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85061176385&partnerID=8YFLogxK

U2 - 10.1177/1094428118825301

DO - 10.1177/1094428118825301

M3 - Article

AN - SCOPUS:85061176385

JO - Organizational Research Methods

JF - Organizational Research Methods

SN - 1094-4281

ER -