Resident Surgical Skills Web-Based Evaluation: A Comparison of 2 Assessment Tools

Ann E. Van Heest, Julie Agel, S. Elizabeth Ames, Ferhan A. Asghar, John J. Harrast, J. Lawrence Marsh, Joshua C. Patt, Robert S. Sterling, Terrance D. Peabody

Research output: Contribution to journalArticlepeer-review

16 Scopus citations


Background:Evaluation of surgical skill competency is necessary as graduate medical education moves toward a competency-based curriculum. This study by the American Board of Orthopaedic Surgery (ABOS) and the Council of Orthopaedic Residency Directors (CORD) compares 2 web-based evaluation tools that assess the level of autonomy that is demonstrated by residents during surgical procedures in the operating room as measured by faculty.Methods:Two hundred and ninety-four residents from 16 orthopaedic surgery residency programs were evaluated by 370 faculty using 2 web-based evaluation tools in a crossover design in which residents requested faculty review of their surgical skills before starting a case. One thousand, one hundred and fifty Ottawa Surgical Competency Operating Room Evaluation (O-Score) assessments, which included a 9-question evaluation of 8 steps of the surgical procedure, were compared with 1,186 P-score evaluations, which included a single-question summative evaluation. Twenty-five different surgical procedures were evaluated.Results:There were no significant differences in rates of resident requests or faculty completion of the 2 scores. The most common surgical procedures that were assessed were total knee arthroplasty (n = 254, 11%), carpal tunnel release (n = 191, 8%), open reduction and internal fixation (ORIF) of stable hip fractures (n = 170, 7%), ORIF of simple ankle fractures (n = 169, 7%), and total hip arthroplasty (n = 166, 7%). Both instruments disclosed significant differences in competency among entry, intermediate, and advanced-level residents. The findings support the construct validity of the evaluation method. The survey results indicated that >70% of the faculty were confident that use of either the P-score or the O-score allowed them to distinguish a resident who can perform the surgery independently from one who needs additional training.Conclusions:This research has led to the modification of the O-score and the P-score into a combined OP-score instrument. The ABOS envisions that the OP-score instrument can be used with an expanded number of surgical procedures as a required element of residency training in the near future.Clinical Relevance:This study allows the profession of orthopaedic surgery education to take a leadership role in the measurement of competence for surgical skills for orthopaedic surgeons in residency training, an important clinically relevant topic to the practice of orthopaedic surgery.

Original languageEnglish (US)
Article numbere18
JournalJournal of Bone and Joint Surgery - American Volume
Issue number5
StatePublished - Mar 6 2019

Bibliographical note

Publisher Copyright:
© 2019 the authors. published by the journal of bone and joint surgery, incorporated. all rights reserved.


Dive into the research topics of 'Resident Surgical Skills Web-Based Evaluation: A Comparison of 2 Assessment Tools'. Together they form a unique fingerprint.

Cite this