Agreement between self-reports and medical records was only fair in a cross-sectional study of performance of annual eye examinations among adults with diabetes in managed care

BACKGROUND: Despite consensus about the importance of measuring quality of diabetes care and the widespread use of self-reports and medical records to assess quality, little is known about the degree of agreement between these data sources. OBJECTIVES: To evaluate agreement between self-reported and medical record data on annual eye examinations and to identify factors associated with agreement. RESEARCH DESIGN AND SUBJECTS: Data from interviews and medical records were available for 8409 adults with diabetes who participated in the baseline round of the Translating Research Into Action for Diabetes (TRIAD) Study. MEASURES: Agreement between self-reports and medical records was evaluated as concordance and Cohen's kappa coefficient. RESULTS: Self-reports indicated a higher performance of annual dilated eye examinations than did medical records (75.9% vs. 38.8%). Concordance between the data sources was 57.9%. Agreement was only fair (kappa coefficient = 0.25; 95% confidence interval, 0.23-0.26). Nearly two-thirds (64.6%) of discordance was due to lack of evidence in the medical record to support self-reported performance of the procedure. After adjustment, agreement was most strongly related to health plan (χ = 977.9, df = 9; P < 0.0001), and remained significantly better for 3 of the 10 health plans (P < 0.00001) and for persons younger than 45 years of age (P = 0.00002). CONCLUSIONS: The low level of agreement between self-report and medical records suggests that many providers of diabetes care do not have easily available accurate information on the eye examination status of their patients.