nlTGCR: A CLASS OF NONLINEAR ACCELERATION PROCEDURES BASED ON CONJUGATE RESIDUALS

Huan He, Ziyuan Tang, Shifan Zhao, Yousef Saad, Yuanzhe Xi

Research output: Contribution to journalArticlepeer-review

2 Scopus citations

Abstract

This paper develops a new class of nonlinear acceleration algorithms based on extending conjugate residual-type procedures from linear to nonlinear equations. The main algorithm has strong similarities with Anderson acceleration as well as with inexact Newton methods-depending on which variant is implemented. We prove theoretically and verify experimentally, on a variety of problems from simulation experiments to deep learning applications, that our method is a powerful accelerated iterative algorithm. The code is available at https://github.com/Data-driven-numerical-methods/Nonlinear-Truncated-Conjugate-Residual.

Original languageEnglish (US)
Pages (from-to)712-743
Number of pages32
JournalSIAM Journal on Matrix Analysis and Applications
Volume45
Issue number1
DOIs
StatePublished - 2024
Externally publishedYes

Bibliographical note

Publisher Copyright:
© 2024 Society for Industrial and Applied Mathematics Publications. All rights reserved.

Keywords

  • Anderson acceleration
  • deep learning
  • generalized conjugate residual
  • Newton's method
  • nonlinear acceleration
  • truncated GCR

Fingerprint

Dive into the research topics of 'nlTGCR: A CLASS OF NONLINEAR ACCELERATION PROCEDURES BASED ON CONJUGATE RESIDUALS'. Together they form a unique fingerprint.

Cite this