TY - JOUR
T1 - Use and Customization of Risk Scores for Predicting Cardiovascular Events Using Electronic Health Record Data
AU - Wolfson, Julian
AU - Vock, David M
AU - Bandyopadhyay, Sunayan
AU - Kottke, Thomas
AU - Vazquez-Benitez, Gabriela
AU - Johnson, Paul
AU - Adomavicius, Gediminas
AU - O'Connor, Patrick J.
N1 - Funding Information:
This work was partially supported by NIH grants R01HL102144, P30DK092924, UL1TR000114, and Agency for Healthcare Research and Quality (AHRQ) grant R21HS017622.
Publisher Copyright:
© 2022 by the authors.
PY - 2017
Y1 - 2017
N2 - Background-—Clinicians who are using the Framingham Risk Score (FRS) or the American College of Cardiology/American Heart Association Pooled Cohort Equations (PCE) to estimate risk for their patients based on electronic health data (EHD) face 4 questions. (1) Do published risk scores applied to EHD yield accurate estimates of cardiovascular risk? (2) Are FRS risk estimates, which are based on data that are up to 45 years old, valid for a contemporary patient population seeking routine care? (3) Do the PCE make the FRS obsolete? (4) Does refitting the risk score using EHD improve the accuracy of risk estimates? Methods and Results-—Data were extracted from the EHD of 84 116 adults aged 40 to 79 years who received care at a large healthcare delivery and insurance organization between 2001 and 2011. We assessed calibration and discrimination for 4 risk scores: Published versions of FRS and PCE and versions obtained by refitting models using a subset of the available EHD. The published FRS was well calibrated (calibration statistic K=9.1, miscalibration ranging from 0% to 17% across risk groups), but the PCE displayed modest evidence of miscalibration (calibration statistic K=43.7, miscalibration from 9% to 31%). Discrimination was similar in both models (C-index=0.740 for FRS, 0.747 for PCE). Refitting the published models using EHD did not substantially improve calibration or discrimination. Conclusions-—We conclude that published cardiovascular risk models can be successfully applied to EHD to estimate cardiovascular risk; the FRS remains valid and is not obsolete; and model refitting does not meaningfully improve the accuracy of risk estimates. (J Am Heart Assoc. 2017;6:e003670. DOI: 10.1161/JAHA.116.003670.).
AB - Background-—Clinicians who are using the Framingham Risk Score (FRS) or the American College of Cardiology/American Heart Association Pooled Cohort Equations (PCE) to estimate risk for their patients based on electronic health data (EHD) face 4 questions. (1) Do published risk scores applied to EHD yield accurate estimates of cardiovascular risk? (2) Are FRS risk estimates, which are based on data that are up to 45 years old, valid for a contemporary patient population seeking routine care? (3) Do the PCE make the FRS obsolete? (4) Does refitting the risk score using EHD improve the accuracy of risk estimates? Methods and Results-—Data were extracted from the EHD of 84 116 adults aged 40 to 79 years who received care at a large healthcare delivery and insurance organization between 2001 and 2011. We assessed calibration and discrimination for 4 risk scores: Published versions of FRS and PCE and versions obtained by refitting models using a subset of the available EHD. The published FRS was well calibrated (calibration statistic K=9.1, miscalibration ranging from 0% to 17% across risk groups), but the PCE displayed modest evidence of miscalibration (calibration statistic K=43.7, miscalibration from 9% to 31%). Discrimination was similar in both models (C-index=0.740 for FRS, 0.747 for PCE). Refitting the published models using EHD did not substantially improve calibration or discrimination. Conclusions-—We conclude that published cardiovascular risk models can be successfully applied to EHD to estimate cardiovascular risk; the FRS remains valid and is not obsolete; and model refitting does not meaningfully improve the accuracy of risk estimates. (J Am Heart Assoc. 2017;6:e003670. DOI: 10.1161/JAHA.116.003670.).
KW - Electronic health record data
KW - risk prediction
KW - risk score
UR - http://www.scopus.com/inward/record.url?scp=85042659208&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85042659208&partnerID=8YFLogxK
U2 - 10.1161/JAHA.116.003670
DO - 10.1161/JAHA.116.003670
M3 - Article
C2 - 28438733
AN - SCOPUS:85042659208
SN - 2047-9980
VL - 6
JO - Journal of the American Heart Association
JF - Journal of the American Heart Association
IS - 4
M1 - e003670
ER -