The relationship between the C-statistic and the accuracy of program-specific evaluations

Andrew Wey, Nicholas Salkowski, Bertram L. Kasiske, Melissa A. Skeans, Sally K. Gustafson, Ajay K Israni, Jon J Snyder

Research output: Contribution to journalArticlepeer-review

13 Scopus citations

Abstract

The C-statistic of the risk-adjustment model is often used to judge the accuracy of program evaluations. However, the C-statistic depends on the variability in risk for individual transplants and may be inappropriate for determining the accuracy of program evaluations. A simulation study investigated the association of the C-statistic with several metrics of program evaluation accuracy, including categorizing programs into the 5-tier system and identifying programs for regulatory review. The simulation study used data from deceased donor kidney-alone transplants for adult recipients in the program-specific reports released January 2018. A range of C-statistics was generated by changing the variability in risk for individual transplants. The C-statistic had no association with any metric of program evaluation accuracy. Instead, the number of expected events at a program was the most important factor. For example, Spearman's rho, which is the correlation of ranks, was −0.27 and −0.72 between the true program-specific hazard ratios and assigned tiers for programs with, respectively, <3 and >10 expected events. Presence of unadjusted risk factors did not modify the associations, although the accuracy of program evaluations was systematically lower. Therefore, the C-statistic provides no information on the accuracy of program evaluations.

Original languageEnglish (US)
Pages (from-to)407-413
Number of pages7
JournalAmerican Journal of Transplantation
Volume19
Issue number2
DOIs
StatePublished - Feb 2019

Bibliographical note

Funding Information:
This work was conducted under the auspices of the Minneapolis Medical Research Foundation, contractor for the Scientific Registry of Transplant Recipients, as a deliverable under contract number HHSH250201500009C (US Department of Health and Human Services, Health Resources and Services Administration, Healthcare Systems Bureau, Division of Transplantation). As a US Government-sponsored work, there are no restrictions on its use. The views expressed herein are those of the authors and not necessarily those of the US Government. AKI was partially supported by R01 HS 24527. The authors thank SRTR colleague Nan Booth, MSW, MPH, ELS, for manuscript editing.

Publisher Copyright:
Published 2018. This article is a U.S. Government work and is in the public domain in the USA

Keywords

  • clinical research/practice
  • organ transplantation in general
  • Scientific Registry for Transplant Recipients (SRTR)
  • statistics

Fingerprint

Dive into the research topics of 'The relationship between the C-statistic and the accuracy of program-specific evaluations'. Together they form a unique fingerprint.

Cite this