The construct and criterion validity of the multi-source feedback process to assess physician performance: A meta-analysis

Ahmed Al Ansari, Tyrone Donnon, Khalid Al Khalifa, Abdulla Darwish, Claudio Violato

Research output: Contribution to journalReview articlepeer-review

18 Scopus citations

Abstract

Background: The purpose of this study was to conduct a meta-analysis on the construct and criterion validity of multi-source feedback (MSF) to assess physicians and surgeons in practice. Methods: In this study, we followed the guidelines for the reporting of observational studies included in a meta-analysis. In addition to PubMed and MEDLINE databases, the CINAHL, EMBASE, and PsycINFO databases were searched from January 1975 to November 2012. All articles listed in the references of the MSF studies were reviewed to ensure that all relevant publications were identified. All 35 articles were independently coded by two authors (AA, TD), and any discrepancies (eg, effect size calculations) were reviewed by the other authors (KA, AD, CV). Results: Physician/surgeon performance measures from 35 studies were identified. A random-effects model of weighted mean effect size differences (d) resulted in: construct validity coefficients for the MSF system on physician/surgeon performance across different levels in practice ranged from d=0.14 (95% confidence interval [CI] 0.40–0.69) to d=1.78 (95% CI 1.20–2.30); construct validity coefficients for the MSF on physician/surgeon performance on two different occasions ranged from d=0.23 (95% CI 0.13–0.33) to d=0.90 (95% CI 0.74–1.10); concurrent validity coefficients for the MSF based on differences in assessor group ratings ranged from d=0.50 (95% CI 0.47–0.52) to d=0.57 (95% CI 0.55–0.60); and predictive validity coefficients for the MSF on physician/surgeon performance across different standardized measures ranged from d=1.28 (95% CI 1.16–1.41) to d=1.43 (95% CI 0.87–2.00). Conclusion: The construct and criterion validity of the MSF system is supported by small to large effect size differences based on the MSF process and physician/surgeon performance across different clinical and nonclinical domain measures.

Original languageEnglish (US)
Pages (from-to)39-51
Number of pages13
JournalAdvances in Medical Education and Practice
Volume5
DOIs
StatePublished - 2014
Externally publishedYes

Bibliographical note

Publisher Copyright:
© 2014 Al Ansari et al.

Keywords

  • Clinical performance
  • Construct validity
  • Criterion validity
  • Meta-analysis
  • Multi-source feedback system

Fingerprint

Dive into the research topics of 'The construct and criterion validity of the multi-source feedback process to assess physician performance: A meta-analysis'. Together they form a unique fingerprint.

Cite this