On orthopedic surgical skill prediction - The limited value of traditional testing

Objectives Primary: to assess the utility of our distal radius fracture repair model as a tool for examining residents' surgical skills. Secondary: to compare the residents' ability to achieve specific biomechanically measured fracture stability with traditional test scores. Design Our laboratory pioneered a model that measures biomechanical qualities of a repaired distal radius fracture. Before participation, all residents to be tested completed specified knowledge examinations. During the laboratory exercise, proctors observed each resident and completed Objective Structured Assessment of Technical Skills forms. At the completion of the laboratory, each specimen was tested biomechanically. Written examinations were completed in a proctored setting and computer examinations at home following the honor system. The laboratory exercise had adequate space and materials and allowed 60 minutes to complete the procedure. Residents had equal access to X-ray imaging. Setting The examination environment of the study resembled an operating room. Participants Postgraduate years 3 and 4 orthopedic residents in our program were asked to participate. The institutional review board reviewed and approved the study as exempt. Results Fracture repair constructs capable of resisting loads expected during rehabilitation were created by approximately half the residents tested. However, traditional written and computer-based testing methods failed to predict which resident's fracture construct would pass the biomechanical testing. Prior in vivo similar case experience was not predictive. Conclusions The idea that "book smart does not equal street smart" applies to the tested model. To measure surgical skill acquisition and increase public safety related to surgery, it will be necessary to employ new and specific examination methods that identify the skill to be acquired and test the acquisition of this skill as precisely as possible.