A predictive and construct validity study of a high-stakes objective clinical examination for assessing the clinical competence of international medical graduates

Background: The purpose of this study was to investigate the predictive and construct validity of a high-stakes objective structured clinical examination (OSCE) used to select candidates for a 3-month clinical rotation to assess practice-readiness status. Summary: Analyses were undertaken to establish the reliability and validity of the OSCE. The generalizability coefficient (Ep ²) for the assessment scores (checklist, global, and total) were all high, ranging from 0.73 to 0.84. Two discriminant analyses (promotion to the 3-month rotation and pass/fail status on the rotation) provided evidence of predictive validity with a 100% correct classification rate in the pass/fail rotation results. Factor analysis results provided evidence of construct validity with four factors identified: Clinical Skills, Internal Medicine, General Medical Knowledge, and Counseling. The known group differences between licensing status and residency experience also provided evidence of construct validity. Conclusions: The results are encouraging for the predictive and construct validity of the OSCE as an assessment of clinical competence.