nlTGCR: A CLASS OF NONLINEAR ACCELERATION PROCEDURES BASED ON CONJUGATE RESIDUALS

Huan He; Ziyuan Tang; Shifan Zhao; Yousef Saad; Yuanzhe Xi

doi:10.1137/23M1576360

nlTGCR: A CLASS OF NONLINEAR ACCELERATION PROCEDURES BASED ON CONJUGATE RESIDUALS

Huan He, Ziyuan Tang, Shifan Zhao, Yousef Saad, Yuanzhe Xi

Research output: Contribution to journal › Article › peer-review

2 Scopus citations

Abstract

This paper develops a new class of nonlinear acceleration algorithms based on extending conjugate residual-type procedures from linear to nonlinear equations. The main algorithm has strong similarities with Anderson acceleration as well as with inexact Newton methods-depending on which variant is implemented. We prove theoretically and verify experimentally, on a variety of problems from simulation experiments to deep learning applications, that our method is a powerful accelerated iterative algorithm. The code is available at https://github.com/Data-driven-numerical-methods/Nonlinear-Truncated-Conjugate-Residual.

Original language	English (US)
Pages (from-to)	712-743
Number of pages	32
Journal	SIAM Journal on Matrix Analysis and Applications
Volume	45
Issue number	1
DOIs	https://doi.org/10.1137/23M1576360
State	Published - 2024
Externally published	Yes

Bibliographical note

Publisher Copyright:
© 2024 Society for Industrial and Applied Mathematics Publications. All rights reserved.

Keywords

Anderson acceleration
deep learning
generalized conjugate residual
Newton's method
nonlinear acceleration
truncated GCR

Publisher link

10.1137/23M1576360

Find It

Find It at U of M Libraries

Cite this

@article{d0d611b1eeff4599a5d64d9c9d056564,

title = "nlTGCR: A CLASS OF NONLINEAR ACCELERATION PROCEDURES BASED ON CONJUGATE RESIDUALS",

abstract = "This paper develops a new class of nonlinear acceleration algorithms based on extending conjugate residual-type procedures from linear to nonlinear equations. The main algorithm has strong similarities with Anderson acceleration as well as with inexact Newton methods-depending on which variant is implemented. We prove theoretically and verify experimentally, on a variety of problems from simulation experiments to deep learning applications, that our method is a powerful accelerated iterative algorithm. The code is available at https://github.com/Data-driven-numerical-methods/Nonlinear-Truncated-Conjugate-Residual.",

keywords = "Anderson acceleration, deep learning, generalized conjugate residual, Newton's method, nonlinear acceleration, truncated GCR",

author = "Huan He and Ziyuan Tang and Shifan Zhao and Yousef Saad and Yuanzhe Xi",

year = "2024",

doi = "10.1137/23M1576360",

language = "English (US)",

volume = "45",

pages = "712--743",

journal = "SIAM Journal on Matrix Analysis and Applications",

issn = "0895-4798",

publisher = "Society for Industrial and Applied Mathematics Publications",

number = "1",

}

TY - JOUR

T1 - nlTGCR

T2 - A CLASS OF NONLINEAR ACCELERATION PROCEDURES BASED ON CONJUGATE RESIDUALS

AU - He, Huan

AU - Tang, Ziyuan

AU - Zhao, Shifan

AU - Saad, Yousef

AU - Xi, Yuanzhe

PY - 2024

Y1 - 2024

N2 - This paper develops a new class of nonlinear acceleration algorithms based on extending conjugate residual-type procedures from linear to nonlinear equations. The main algorithm has strong similarities with Anderson acceleration as well as with inexact Newton methods-depending on which variant is implemented. We prove theoretically and verify experimentally, on a variety of problems from simulation experiments to deep learning applications, that our method is a powerful accelerated iterative algorithm. The code is available at https://github.com/Data-driven-numerical-methods/Nonlinear-Truncated-Conjugate-Residual.

AB - This paper develops a new class of nonlinear acceleration algorithms based on extending conjugate residual-type procedures from linear to nonlinear equations. The main algorithm has strong similarities with Anderson acceleration as well as with inexact Newton methods-depending on which variant is implemented. We prove theoretically and verify experimentally, on a variety of problems from simulation experiments to deep learning applications, that our method is a powerful accelerated iterative algorithm. The code is available at https://github.com/Data-driven-numerical-methods/Nonlinear-Truncated-Conjugate-Residual.

KW - Anderson acceleration

KW - deep learning

KW - generalized conjugate residual

KW - Newton's method

KW - nonlinear acceleration

KW - truncated GCR

UR - http://www.scopus.com/inward/record.url?scp=85186895213&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85186895213&partnerID=8YFLogxK

U2 - 10.1137/23M1576360

DO - 10.1137/23M1576360

M3 - Article

AN - SCOPUS:85186895213

SN - 0895-4798

VL - 45

SP - 712

EP - 743

JO - SIAM Journal on Matrix Analysis and Applications

JF - SIAM Journal on Matrix Analysis and Applications

IS - 1

ER -

nlTGCR: A CLASS OF NONLINEAR ACCELERATION PROCEDURES BASED ON CONJUGATE RESIDUALS

Abstract

Bibliographical note

Keywords

Publisher link

Find It

Other files and links

Fingerprint

Cite this