TY - JOUR

T1 - Error analysis of conventional discrete and gradient dynamic programming

AU - Kitanidis, Peter K.

AU - Foufoula‐Georgiou, Efi

PY - 1987/5

Y1 - 1987/5

N2 - An asymptotic error analysis of the conventional discrete dynamic programming (DDP) method is presented, and upper bounds of the error in the control policy (i.e., the difference of the estimated and true optimal control) at each operation period are computed. This error is shown to be of the order of the state discretization interval (ΔS), a result with significant implications in the optimization of multistate systems where the “curse of dimensionality” restricts the number of states to a relatively small number. The error in the optimal cost varies with ΔS2. The analysis provides useful insights into the effects of state discretization on calculated control and cost functions, the comparability of results from different discretizations, and criteria about the required number of nodes. In an effort to reduce the discretization error in the case of smooth cost functions, a new discrete dynamic programming method, termed gradient dynamic programming (GDP), is proposed. GDP uses a piecewise Hermite interpolation of the cost‐to‐go function, at each stage, which preserves the values of the cost‐to‐go function and of its first derivatives at the discretization nodes. The error in the control policy is shown to be of the order of (ΔS)3 and the error in the cost to vary with ΔS4. Thus as ΔS decreases, GDP converges to the true optimum much more rapidly than DDP. Another major advantage of the new methodology is that it facilitates the use of Newton‐type iterative methods in the solution of the nonlinear optimization problems at each stage. The linear convergence of DDP and the superlinear convergence of GDP are illustrated in an example.

AB - An asymptotic error analysis of the conventional discrete dynamic programming (DDP) method is presented, and upper bounds of the error in the control policy (i.e., the difference of the estimated and true optimal control) at each operation period are computed. This error is shown to be of the order of the state discretization interval (ΔS), a result with significant implications in the optimization of multistate systems where the “curse of dimensionality” restricts the number of states to a relatively small number. The error in the optimal cost varies with ΔS2. The analysis provides useful insights into the effects of state discretization on calculated control and cost functions, the comparability of results from different discretizations, and criteria about the required number of nodes. In an effort to reduce the discretization error in the case of smooth cost functions, a new discrete dynamic programming method, termed gradient dynamic programming (GDP), is proposed. GDP uses a piecewise Hermite interpolation of the cost‐to‐go function, at each stage, which preserves the values of the cost‐to‐go function and of its first derivatives at the discretization nodes. The error in the control policy is shown to be of the order of (ΔS)3 and the error in the cost to vary with ΔS4. Thus as ΔS decreases, GDP converges to the true optimum much more rapidly than DDP. Another major advantage of the new methodology is that it facilitates the use of Newton‐type iterative methods in the solution of the nonlinear optimization problems at each stage. The linear convergence of DDP and the superlinear convergence of GDP are illustrated in an example.

UR - http://www.scopus.com/inward/record.url?scp=0023346723&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0023346723&partnerID=8YFLogxK

U2 - 10.1029/WR023i005p00845

DO - 10.1029/WR023i005p00845

M3 - Article

AN - SCOPUS:0023346723

SN - 0043-1397

VL - 23

SP - 845

EP - 858

JO - Water Resources Research

JF - Water Resources Research

IS - 5

ER -