Predicting Performance of First-Year Residents: Correlations between Structured Interview, Licensure Exam, and Competency Scores in a Multi-Institutional Study

Brittany Marcus-Blank, Jeffrey A. Dahlke, Jonathan P Braman, Emily C Borman-Shoap, Ezgi Tiryaki, Jeffrey G Chipman, John S. Andrews, Paul R Sackett, Michael J. Cullen

Research output: Contribution to journalArticlepeer-review

9 Scopus citations

Abstract

Purpose To determine whether scores on structured interview (SI) questions designed to measure noncognitive competencies in physicians (1) predict subsequent first-year resident performance on Accreditation Council for Graduate Medical Education (ACGME) milestones and (2) add incremental validity over United States Medical Licensing Examination (USMLE) Step 1 and Step 2 Clinical Knowledge scores in predicting performance. Method The authors developed 18 behavioral description questions to measure key noncognitive competencies (e.g., teamwork). In 2013-2015, 14 programs (13 residency, 1 fellowship) from 6 institutions used subsets of these questions in their selection processes. The authors conducted analyses to determine the validity of SI and USMLE scores in predicting first-year resident milestone performance in the ACGME's core competency domains and overall. Results SI scores predicted midyear and year-end overall performance (r = 0.18 and 0.19, respectively, P <.05) and year-end performance on patient care, interpersonal and communication skills, and professionalism competencies (r = 0.23, r = 0.22, and r = 0.20, respectively, P <.05). SI scores contributed incremental validity over USMLE scores in predicting year-end performance on patient care (ΔR = 0.05), interpersonal and communication skills (ΔR = 0.09), and professionalism (ΔR = 0.09; all P <.05). USMLE scores contributed incremental validity over SI scores in predicting year-end performance overall and on patient care and medical knowledge. Conclusions SI scores predict first-year resident year-end performance in the interpersonal and communication skills, patient care, and professionalism competency domains. Future research should investigate whether SIs predict a range of clinically relevant outcomes.

Original languageEnglish (US)
Pages (from-to)378-387
Number of pages10
JournalAcademic Medicine
Volume94
Issue number3
DOIs
StatePublished - Mar 1 2019

PubMed: MeSH publication types

  • Journal Article
  • Multicenter Study

Fingerprint Dive into the research topics of 'Predicting Performance of First-Year Residents: Correlations between Structured Interview, Licensure Exam, and Competency Scores in a Multi-Institutional Study'. Together they form a unique fingerprint.

Cite this