TY - JOUR

T1 - Improved training of physics-informed neural networks for parabolic differential equations with sharply perturbed initial conditions

AU - Zong, Yifei

AU - He, Qi Zhi

AU - Tartakovsky, Alexandre M.

N1 - Publisher Copyright:
© 2023 Elsevier B.V.

PY - 2023/9/1

Y1 - 2023/9/1

N2 - We propose a multi-component approach for improving the training of the physics-informed neural network (PINN) model for parabolic problems with a sharply perturbed initial condition. As an example of a parabolic problem, we consider the advection–dispersion equation (ADE) with a point (Gaussian) source initial condition. In the d-dimensional ADE, perturbations in the initial condition decay with time t as t−d/2. We demonstrate that for d≥2, this decay rate can cause a large approximation error in the PINN solution. Furthermore, localized large gradients in the ADE solution make the (common in PINN) Latin hypercube sampling of the equation's residual highly inefficient. Finally, the PINN solution of parabolic equations is sensitive to the choice of weights in the loss function. We propose a normalized form of ADE where the initial perturbation of the solution does not decrease in amplitude and demonstrate that this normalization significantly reduces the PINN approximation error. Next, we present an adaptive sampling scheme based on the analytical estimate of the solution decay rate that significantly reduces the PINN estimation error for the same number of sampling (residual) points. Finally, we develop criteria for selecting weights based on the order of magnitude of different terms in the loss function. We demonstrate the accuracy of the proposed PINN model for forward, inverse, and backward ADEs.

AB - We propose a multi-component approach for improving the training of the physics-informed neural network (PINN) model for parabolic problems with a sharply perturbed initial condition. As an example of a parabolic problem, we consider the advection–dispersion equation (ADE) with a point (Gaussian) source initial condition. In the d-dimensional ADE, perturbations in the initial condition decay with time t as t−d/2. We demonstrate that for d≥2, this decay rate can cause a large approximation error in the PINN solution. Furthermore, localized large gradients in the ADE solution make the (common in PINN) Latin hypercube sampling of the equation's residual highly inefficient. Finally, the PINN solution of parabolic equations is sensitive to the choice of weights in the loss function. We propose a normalized form of ADE where the initial perturbation of the solution does not decrease in amplitude and demonstrate that this normalization significantly reduces the PINN approximation error. Next, we present an adaptive sampling scheme based on the analytical estimate of the solution decay rate that significantly reduces the PINN estimation error for the same number of sampling (residual) points. Finally, we develop criteria for selecting weights based on the order of magnitude of different terms in the loss function. We demonstrate the accuracy of the proposed PINN model for forward, inverse, and backward ADEs.

KW - Backward advection–dispersion equations

KW - Deep neural network training

KW - Importance sampling

KW - Inverse problems

KW - Parabolic equations

KW - Physics-informed neural networks

UR - http://www.scopus.com/inward/record.url?scp=85163299655&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85163299655&partnerID=8YFLogxK

U2 - 10.1016/j.cma.2023.116125

DO - 10.1016/j.cma.2023.116125

M3 - Article

AN - SCOPUS:85163299655

SN - 0045-7825

VL - 414

JO - Computer Methods in Applied Mechanics and Engineering

JF - Computer Methods in Applied Mechanics and Engineering

M1 - 116125

ER -