Assessment of validity of an OSATS for cystoscopic and ureteroscopic cognitive and psychomotor skills

Purpose: The purpose of the study was to assess the construct validity of an Objective Structured Assessment of Technical Skills (OSATS) developed for cystoscopic and ureteroscopic cognitive and psychomotor skills. Materials and Methods: An OSATS was designed based on a 14-point comprehensive curriculum prepared by two experts that targeted both cognitive and psychomotor cystoscopic and ureteroscopic skills. Ten urology residents from a single institution with different levels of training were assessed on a series of stations that targeted these skills. Evaluation of cognitive skills was done via a written examination, and psychomotor skills assessment was done by experts using both subjective and objective metrics. Results: Twelve of 15 cognitive tasks and 5of 5 psychomotor tasks demonstrated construct validity with correlation coefficient (r) more than .75. All three of the cognitive tasks that failed to initially demonstrate validity did so on editorial revision and restructuring of the questions. Conclusion: Our cystoscopic and ureteroscopic OSATS showed excellent construct validity for our population of residents, and we have incorporated it into our urologic skills curriculum.