Improved training of physics-informed neural networks for parabolic differential equations with sharply perturbed initial conditions

We propose a multi-component approach for improving the training of the physics-informed neural network (PINN) model for parabolic problems with a sharply perturbed initial condition. As an example of a parabolic problem, we consider the advection–dispersion equation (ADE) with a point (Gaussian) source initial condition. In the d-dimensional ADE, perturbations in the initial condition decay with time t as t^−d/2. We demonstrate that for d≥2, this decay rate can cause a large approximation error in the PINN solution. Furthermore, localized large gradients in the ADE solution make the (common in PINN) Latin hypercube sampling of the equation's residual highly inefficient. Finally, the PINN solution of parabolic equations is sensitive to the choice of weights in the loss function. We propose a normalized form of ADE where the initial perturbation of the solution does not decrease in amplitude and demonstrate that this normalization significantly reduces the PINN approximation error. Next, we present an adaptive sampling scheme based on the analytical estimate of the solution decay rate that significantly reduces the PINN estimation error for the same number of sampling (residual) points. Finally, we develop criteria for selecting weights based on the order of magnitude of different terms in the loss function. We demonstrate the accuracy of the proposed PINN model for forward, inverse, and backward ADEs.