Representativeness of Workplace-Based Operative Performance Assessments for Resident Operative Experience

Kenneth L. Abbott, Andrew E. Krumm, Michael J. Clark, Daniel E. Kendrick, Jesse K. Kelley, Brian C. George

Research output: Contribution to journalArticlepeer-review

Abstract

OBJECTIVE: Workplace-based assessment is increasingly prevalent in surgical education, especially for assessing operative skill. With current implementations, not all observed clinical performances are assessed, in part because trainees often have discretion about when they seek assessment. As a result, these samples of observed operative performances may not be representative of the full breadth of experience of surgical trainees. Therefore, analyses of these samples may be biased. We aimed to benchmark patterns of procedures logged in the SIMPL operative performance assessment system against records of trainee experience in Accreditation Council for Graduate Medical Education (ACGME) case logs. DESIGN: We analyzed SIMPL longitudinal intraoperative performance assessments from categorical trainees in US general surgery residency programs. We compared overall patterns of how procedures are logged in SIMPL and in ACGME case logs using a Pearson correlation, and we examined differences in how individual procedures are logged in each system using Fisher's exact test. RESULTS: Total procedure frequency from the SIMPL dataset was strongly correlated with total procedure frequency from ACGME case logs (r = 0.86, 95% CI 0.80-0.90). A subset of these procedures (10 of 116 procedures) was logged more frequently in the SIMPL dataset. These 10 procedures accounted for 56% of SIMPL observations and 30% of ACGME logged cases. Case complexity was comparable for assessments initiated by residents and faculty. CONCLUSIONS: Samples of intraoperative performance ratings gathered using the SIMPL application largely resemble ACGME case logs. There is no evidence to indicate that residents preferentially select fewer complex cases for assessment.

Original languageEnglish (US)
Pages (from-to)769-774
Number of pages6
JournalJournal of surgical education
Volume79
Issue number3
DOIs
StatePublished - May 1 2022

Bibliographical note

Funding Information:
The authors thank the program directors, coordinators, faculty, and residents from the programs who are members of the Society for Improving Medical Professional Learning (SIMPL). Without their educational efforts and ongoing support for SIMPL, this study would not have been possible.

Publisher Copyright:
© 2021 Association of Program Directors in Surgery

Keywords

  • ACGME
  • SIMPL
  • case logs
  • performance assessment
  • resident
  • workplace-based assessment

Fingerprint

Dive into the research topics of 'Representativeness of Workplace-Based Operative Performance Assessments for Resident Operative Experience'. Together they form a unique fingerprint.

Cite this