Can a Situational Judgment Test Identify Trainees at Risk of Professionalism Issues? A Multi-Institutional, Prospective Cohort Study

Michael J Cullen, Charlene Zhang, Paul R. Sackett, Krima Thakker, John Q. Young

Research output: Contribution to journalArticlepeer-review

1 Scopus citations

Abstract

PURPOSE: To determine whether overall situational judgment test (SJT) scores are associated with programs' clinical competency committee (CCC) ratings of trainee professionalism, any concerning behavior, and concerning behavior requiring active remediation at 2 time periods.

METHOD: In fall 2019, trainees from 17 U.S. programs (16 residency, 1 fellowship) took an online 15-scenario SJT developed to measure 7 dimensions of professionalism. CCC midyear and year-end (6 months and 1 year following SJT completion, respectively) professionalism scores and concern ratings were gathered for academic year 2019-2020. Analyses were conducted to determine whether overall SJT scores related to overall professionalism ratings, trainees displaying any concerns, and trainees requiring active remediation at both time periods.

RESULTS: Overall SJT scores correlated positively with midyear and year-end overall professionalism ratings ( r = .21 and .14, P < .001 and = .03, respectively). Holding gender and race/ethnicity constant, a 1 standard deviation (SD) increase in overall SJT score was associated with a .20 SD increase in overall professionalism ratings at midyear ( P = .005) and a .22 SD increase at year-end ( P = .001). Holding gender and race/ethnicity constant, a 1 SD increase in overall SJT score decreased the odds of a trainee displaying any concerns by 37% (odds ratio [OR] 95% confidence interval [CI]: [.44, .87], P = .006) at midyear and 34% (OR 95% CI: [.46, .95], P = .025) at year-end and decreased the odds of a trainee requiring active remediation by 51% (OR 95% CI: [.25, .90], P = .02) at midyear.

CONCLUSIONS: Overall SJT scores correlated positively with midyear and year-end overall professionalism ratings and were associated with whether trainees exhibited any concerning behavior at midyear and year-end and whether trainees needed active remediation at midyear. Future research should investigate whether other potential professionalism measures are associated with concerning trainee behavior.

Original languageEnglish (US)
Pages (from-to)1494-1503
Number of pages10
JournalAcademic Medicine
Volume97
Issue number10
DOIs
StatePublished - Oct 1 2022

Bibliographical note

Publisher Copyright:
© 2022 Lippincott Williams and Wilkins. All rights reserved.

Keywords

  • Clinical Competence
  • Humans
  • Internship and Residency
  • Judgment
  • Professionalism
  • Prospective Studies

PubMed: MeSH publication types

  • Multicenter Study
  • Journal Article

Fingerprint

Dive into the research topics of 'Can a Situational Judgment Test Identify Trainees at Risk of Professionalism Issues? A Multi-Institutional, Prospective Cohort Study'. Together they form a unique fingerprint.

Cite this