Machine learning algorithm to predict anterior cruciate ligament revision demonstrates external validity

R. Kyle Martin, Solvejg Wastvedt, Ayoosh Pareek, Andreas Persson, Håvard Visnes, Anne Marie Fenstad, Gilbert Moatshe, Julian Wolfson, Martin Lind, Lars Engebretsen

Research output: Contribution to journalArticlepeer-review

26 Scopus citations

Abstract

Purpose: External validation of machine learning predictive models is achieved through evaluation of model performance on different groups of patients than were used for algorithm development. This important step is uncommonly performed, inhibiting clinical translation of newly developed models. Machine learning analysis of the Norwegian Knee Ligament Register (NKLR) recently led to the development of a tool capable of estimating the risk of anterior cruciate ligament (ACL) revision (https://swastvedt.shinyapps.io/calculator_rev/). The purpose of this study was to determine the external validity of the NKLR model by assessing algorithm performance when applied to patients from the Danish Knee Ligament Registry (DKLR). Methods: The primary outcome measure of the NKLR model was probability of revision ACL reconstruction within 1, 2, and/or 5 years. For external validation, all DKLR patients with complete data for the five variables required for NKLR prediction were included. The five variables included graft choice, femur fixation device, KOOS QOL score at surgery, years from injury to surgery, and age at surgery. Predicted revision probabilities were calculated for all DKLR patients. The model performance was assessed using the same metrics as the NKLR study: concordance and calibration. Results: In total, 10,922 DKLR patients were included for analysis. Average follow-up time or time-to-revision was 8.4 (± 4.3) years and overall revision rate was 6.9%. Surgical technique trends (i.e., graft choice and fixation devices) and injury characteristics (i.e., concomitant meniscus and cartilage pathology) were dissimilar between registries. The model produced similar concordance when applied to the DKLR population compared to the original NKLR test data (DKLR: 0.68; NKLR: 0.68–0.69). Calibration was poorer for the DKLR population at one and five years post primary surgery but similar to the NKLR at two years. Conclusion: The NKLR machine learning algorithm demonstrated similar performance when applied to patients from the DKLR, suggesting that it is valid for application outside of the initial patient population. This represents the first machine learning model for predicting revision ACL reconstruction that has been externally validated. Clinicians can use this in-clinic calculator to estimate revision risk at a patient specific level when discussing outcome expectations pre-operatively. While encouraging, it should be noted that the performance of the model on patients undergoing ACL reconstruction outside of Scandinavia remains unknown. Level of evidence: III.

Original languageEnglish (US)
Pages (from-to)368-375
Number of pages8
JournalKnee Surgery, Sports Traumatology, Arthroscopy
Volume30
Issue number2
DOIs
StatePublished - Feb 2022

Bibliographical note

Funding Information:
This study was funded by a Norwegian Centennial Chair seed grant. Funding supported the machine learning analysis and interpretation. The funding agencies had no direct role in the investigation.

Publisher Copyright:
© 2021, The Author(s).

Keywords

  • ACL Reconstruction
  • ACL revision
  • Artificial intelligence
  • Machine learning
  • Outcome prediction
  • Reoperation
  • Humans
  • Anterior Cruciate Ligament/surgery
  • Quality of Life
  • Registries
  • Anterior Cruciate Ligament Injuries/diagnosis
  • Machine Learning

PubMed: MeSH publication types

  • Journal Article

Fingerprint

Dive into the research topics of 'Machine learning algorithm to predict anterior cruciate ligament revision demonstrates external validity'. Together they form a unique fingerprint.

Cite this