Narrative Performance Level Assignments at Initial Entrustment and Graduation: Integrating EPAs and Milestones to Improve Learner Assessment

Daniel J. Schumacher, Alan Schwartz, Joseph A. Zenel, Nicole Paradise Black, Keith Ponitz, Robyn Blair, Christin M. Traba, Sue Poynter, Beth King, Robert Englander, Adam Rosenberg, Dilipkumar Patel, Candace Smith-King, Meghan O'Connor, Javier Gonzalez Del Rey, Suzanne Lavoie, Emily Borman-Shoap, Carol Carraccio

Research output: Contribution to journalArticlepeer-review

12 Scopus citations


Purpose To determine which narrative performance level for each general pediatrics entrustable professional activity (EPA) reflects the minimum level clinical competency committees (CCCs) felt should be associated with graduation as well as initial entrustment and compare expected narrative performance levels (ENPLs) for each EPA with actual narrative performance levels (ANPLs) assigned to residents at initial entrustment. Method A series of 5 narratives, corresponding to the 5 milestone performance levels, were developed for each of the 17 general pediatrics EPAs. In academic year (AY) 2015-2016, the CCCs at 22 Association of Pediatric Program Directors Longitudinal Educational Assessment Research Network member sites reported ENPLs for initial entrustment and at time of graduation. From AYs 2015-2016 to 2017-2018, programs reported ANPLs for initial entrustment decisions. ENPLs and ANPLs were compared using a logistic mixed effects model. Results ENPLs for graduation and entrustment were most often level 3 (competent) followed by level 4 (proficient). For 8 EPAs, the ENPLs for graduation and entrustment were the same. For the remaining 9, some programs would entrust residents before graduation or graduate them before entrusting them. There were 4,266 supervision level reports for initial entrustment for which an ANPL was provided. ANPLs that were lower than the ENPLs were significantly more likely to be assigned to the medical home-well child (OR = 0.39; 95% CI: 0.26-0.57), transition to adult care (OR = 0.43; 95% CI: 0.19-0.95), behavioral or mental health (OR = 0.36; 95% CI: 0.18-0.71), make referrals (OR = 0.31; 95% CI: 0.17-0.55), lead a team (OR = 0.34; 95% CI: 0.22-0.52), and handovers (OR = 0.18; 95% CI: 0.09-0.36) EPAs. Conclusions CCCs reported lower ENPLs for graduation than for entrustment for 5 EPAs, possibly indicating curricular gaps that milestones and EPAs could help identify.

Original languageEnglish (US)
Pages (from-to)1736-1744
Number of pages9
JournalAcademic Medicine
Issue number11
StatePublished - Nov 1 2020

Bibliographical note

Publisher Copyright:
© 2020 Lippincott Williams and Wilkins. All rights reserved.


Dive into the research topics of 'Narrative Performance Level Assignments at Initial Entrustment and Graduation: Integrating EPAs and Milestones to Improve Learner Assessment'. Together they form a unique fingerprint.

Cite this