TY - JOUR
T1 - Changes in performance
T2 - A 5-year longitudinal study of participants in a multi-source feedback programme
AU - Violato, Claudio
AU - Lockyer, Jocelyn M.
AU - Fidler, Herta
PY - 2008/10/1
Y1 - 2008/10/1
N2 - Objectives: Multi-source feedback (MSF) enables performance data to be provided to doctors from patients, co-workers and medical colleagues. This study examined the evidence for the validity of MSF instruments for general practice, investigated changes in performance for doctors who participated twice, 5 years apart, and determined the association between change in performance and initial assessment and socio-demographic characteristics. Methods: Data for 250 doctors included three datasets per doctor from, respectively, 25 patients, eight co-workers and eight medical colleagues, collected on two occasions. Results: There was high internal consistency (α > 0.90) and adequate generalisability (Ep2 > 0.70). D study results indicate adequate generalisability coefficients for groups of eight assessors (medical colleagues, co-workers) and 25 patient surveys. Confirmatory factor analyses provided evidence for the validity of factors that were theoretically expected, meaningful and cohesive. Comparative fit indices were 0.91 for medical colleague data, 0.87 for co-worker data and 0.81 for patient data. Paired t-test analysis showed significant change between the two assessments from medical colleagues and co-workers, but not between the two patient surveys. Multiple linear regressions explained 2.1% of the variance at time 2 for medical colleagues, 21.4% of the variance for co-workers and 16.35% of the variance for patient assessments, with professionalism a key variable in all regressions. Conclusions: There is evidence for the construct validity of the instruments and for their stability over time. Upward changes in performance will occur, although their effect size is likely to be small to moderate.
AB - Objectives: Multi-source feedback (MSF) enables performance data to be provided to doctors from patients, co-workers and medical colleagues. This study examined the evidence for the validity of MSF instruments for general practice, investigated changes in performance for doctors who participated twice, 5 years apart, and determined the association between change in performance and initial assessment and socio-demographic characteristics. Methods: Data for 250 doctors included three datasets per doctor from, respectively, 25 patients, eight co-workers and eight medical colleagues, collected on two occasions. Results: There was high internal consistency (α > 0.90) and adequate generalisability (Ep2 > 0.70). D study results indicate adequate generalisability coefficients for groups of eight assessors (medical colleagues, co-workers) and 25 patient surveys. Confirmatory factor analyses provided evidence for the validity of factors that were theoretically expected, meaningful and cohesive. Comparative fit indices were 0.91 for medical colleague data, 0.87 for co-worker data and 0.81 for patient data. Paired t-test analysis showed significant change between the two assessments from medical colleagues and co-workers, but not between the two patient surveys. Multiple linear regressions explained 2.1% of the variance at time 2 for medical colleagues, 21.4% of the variance for co-workers and 16.35% of the variance for patient assessments, with professionalism a key variable in all regressions. Conclusions: There is evidence for the construct validity of the instruments and for their stability over time. Upward changes in performance will occur, although their effect size is likely to be small to moderate.
KW - feedback
KW - Alberta
KW - Clinical competence/standards
KW - Family practice/standards
KW - Longitudinal studies
KW - Physicians, family/standards
KW - Psychometrics
UR - http://www.scopus.com/inward/record.url?scp=51649117156&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=51649117156&partnerID=8YFLogxK
U2 - 10.1111/j.1365-2923.2008.03127.x
DO - 10.1111/j.1365-2923.2008.03127.x
M3 - Article
C2 - 18823520
AN - SCOPUS:51649117156
SN - 0308-0110
VL - 42
SP - 1007
EP - 1013
JO - Medical education
JF - Medical education
IS - 10
ER -