Ceiling Effect of the Combined Norwegian and Danish Knee Ligament Registers Limits Anterior Cruciate Ligament Reconstruction Outcome Prediction

R. Kyle Martin, Solvejg Wastvedt, Ayoosh Pareek, Andreas Persson, Håvard Visnes, Anne Marie Fenstad, Gilbert Moatshe, Julian Wolfson, Martin Lind, Lars Engebretsen

Research output: Contribution to journalArticlepeer-review

6 Scopus citations


Background: Clinical tools based on machine learning analysis now exist for outcome prediction after primary anterior cruciate ligament reconstruction (ACLR). Relying partly on data volume, the general principle is that more data may lead to improved model accuracy. Purpose/Hypothesis: The purpose was to apply machine learning to a combined data set from the Norwegian and Danish knee ligament registers (NKLR and DKRR, respectively), with the aim of producing an algorithm that can predict revision surgery with improved accuracy relative to a previously published model developed using only the NKLR. The hypothesis was that the additional patient data would result in an algorithm that is more accurate. Study Design: Cohort study; Level of evidence, 3. Methods: Machine learning analysis was performed on combined data from the NKLR and DKRR. The primary outcome was the probability of revision ACLR within 1, 2, and 5 years. Data were split randomly into training sets (75%) and test sets (25%). There were 4 machine learning models examined: Cox lasso, random survival forest, gradient boosting, and super learner. Concordance and calibration were calculated for all 4 models. Results: The data set included 62,955 patients in which 5% underwent a revision surgical procedure with a mean follow-up of 7.6 ± 4.5 years. The 3 nonparametric models (random survival forest, gradient boosting, and super learner) performed best, demonstrating moderate concordance (0.67 [95% CI, 0.64-0.70]), and were well calibrated at 1 and 2 years. Model performance was similar to that of the previously published model (NKLR-only model: concordance, 0.67-0.69; well calibrated). Conclusion: Machine learning analysis of the combined NKLR and DKRR enabled prediction of the revision ACLR risk with moderate accuracy. However, the resulting algorithms were less user-friendly and did not demonstrate superior accuracy in comparison with the previously developed model based on patients from the NKLR alone, despite the analysis of nearly 63,000 patients. This ceiling effect suggests that simply adding more patients to current national knee ligament registers is unlikely to improve predictive capability and may prompt future changes to increase variable inclusion.

Original languageEnglish (US)
Pages (from-to)2324-2332
Number of pages9
JournalAmerican Journal of Sports Medicine
Issue number9
StatePublished - Jul 2023

Bibliographical note

Funding Information:
One or more of the authors has declared the following potential conflict of interest or source of funding: This study was funded by a Norwegian Centennial Chair seed grant. R.K.M. has received consulting fees from Smith & Nephew and support for education from Gemini/Arthrex. G.M. has received consulting fees from Arthrex and IBSA. M.L. has received consulting fees from Smith & Nephew. L.E. has received research support from Biomet and Health South-Eastern Norway and royalties from Arthrex and Smith & Nephew. AOSSM checks author disclosures against the Open Payments Database (OPD). AOSSM has not conducted an independent investigation on the OPD and disclaims any liability or responsibility relating thereto.

Publisher Copyright:
© 2023 The Author(s).


  • ACL revision
  • artificial intelligence
  • machine learning
  • outcome prediction

PubMed: MeSH publication types

  • Journal Article
  • Research Support, Non-U.S. Gov't


Dive into the research topics of 'Ceiling Effect of the Combined Norwegian and Danish Knee Ligament Registers Limits Anterior Cruciate Ligament Reconstruction Outcome Prediction'. Together they form a unique fingerprint.

Cite this