Computing Stabilizing Linear Controllers via Policy Iteration

Andrew Lamperski

doi:10.1109/CDC42340.2020.9304202

Computing Stabilizing Linear Controllers via Policy Iteration

Andrew Lamperski

Electrical and Computer Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

31 Scopus citations

Abstract

In recent years, a wide number of theoretical papers have focused on reinforcement learning approaches to the linear quadratic regulator (LQR) problem. However, nearly all of these papers assume that an initial stabilizing controller is given. This paper gives a model-free, off-policy reinforcement learning algorithm for computing a stabilizing controller for deterministic LQR problems with unknown dynamics and cost matrices. When the system is stabilizable, a controller which is guaranteed to stabilize the system is computed after finitely many steps. Furthermore, the solution converges to the optimal LQR gain.

Original language	English (US)
Title of host publication	2020 59th IEEE Conference on Decision and Control, CDC 2020
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	1902-1907
Number of pages	6
ISBN (Electronic)	9781728174471
DOIs	https://doi.org/10.1109/CDC42340.2020.9304202
State	Published - Dec 14 2020
Event	59th IEEE Conference on Decision and Control, CDC 2020 - Virtual, Jeju Island, Korea, Republic of Duration: Dec 14 2020 → Dec 18 2020

Publication series

Name	Proceedings of the IEEE Conference on Decision and Control
Volume	2020-December
ISSN (Print)	0743-1546
ISSN (Electronic)	2576-2370

Conference

Conference	59th IEEE Conference on Decision and Control, CDC 2020
Country/Territory	Korea, Republic of
City	Virtual, Jeju Island
Period	12/14/20 → 12/18/20

Bibliographical note

Publisher Copyright:
© 2020 IEEE.

Publisher link

10.1109/CDC42340.2020.9304202

Find It

Find It at U of M Libraries

Cite this

Computing Stabilizing Linear Controllers via Policy Iteration. / Lamperski, Andrew.
2020 59th IEEE Conference on Decision and Control, CDC 2020. Institute of Electrical and Electronics Engineers Inc., 2020. p. 1902-1907 9304202 (Proceedings of the IEEE Conference on Decision and Control; Vol. 2020-December).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Lamperski, A 2020, Computing Stabilizing Linear Controllers via Policy Iteration. in 2020 59th IEEE Conference on Decision and Control, CDC 2020., 9304202, Proceedings of the IEEE Conference on Decision and Control, vol. 2020-December, Institute of Electrical and Electronics Engineers Inc., pp. 1902-1907, 59th IEEE Conference on Decision and Control, CDC 2020, Virtual, Jeju Island, Korea, Republic of, 12/14/20. https://doi.org/10.1109/CDC42340.2020.9304202

@inproceedings{aa4fb9f51341413c940e01930f58b47d,

title = "Computing Stabilizing Linear Controllers via Policy Iteration",

abstract = "In recent years, a wide number of theoretical papers have focused on reinforcement learning approaches to the linear quadratic regulator (LQR) problem. However, nearly all of these papers assume that an initial stabilizing controller is given. This paper gives a model-free, off-policy reinforcement learning algorithm for computing a stabilizing controller for deterministic LQR problems with unknown dynamics and cost matrices. When the system is stabilizable, a controller which is guaranteed to stabilize the system is computed after finitely many steps. Furthermore, the solution converges to the optimal LQR gain.",

author = "Andrew Lamperski",

note = "Publisher Copyright: {\textcopyright} 2020 IEEE.; 59th IEEE Conference on Decision and Control, CDC 2020 ; Conference date: 14-12-2020 Through 18-12-2020",

year = "2020",

month = dec,

day = "14",

doi = "10.1109/CDC42340.2020.9304202",

language = "English (US)",

series = "Proceedings of the IEEE Conference on Decision and Control",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "1902--1907",

booktitle = "2020 59th IEEE Conference on Decision and Control, CDC 2020",

}

TY - GEN

T1 - Computing Stabilizing Linear Controllers via Policy Iteration

AU - Lamperski, Andrew

PY - 2020/12/14

Y1 - 2020/12/14

N2 - In recent years, a wide number of theoretical papers have focused on reinforcement learning approaches to the linear quadratic regulator (LQR) problem. However, nearly all of these papers assume that an initial stabilizing controller is given. This paper gives a model-free, off-policy reinforcement learning algorithm for computing a stabilizing controller for deterministic LQR problems with unknown dynamics and cost matrices. When the system is stabilizable, a controller which is guaranteed to stabilize the system is computed after finitely many steps. Furthermore, the solution converges to the optimal LQR gain.

AB - In recent years, a wide number of theoretical papers have focused on reinforcement learning approaches to the linear quadratic regulator (LQR) problem. However, nearly all of these papers assume that an initial stabilizing controller is given. This paper gives a model-free, off-policy reinforcement learning algorithm for computing a stabilizing controller for deterministic LQR problems with unknown dynamics and cost matrices. When the system is stabilizable, a controller which is guaranteed to stabilize the system is computed after finitely many steps. Furthermore, the solution converges to the optimal LQR gain.

UR - https://www.scopus.com/pages/publications/85099880545

UR - https://www.scopus.com/pages/publications/85099880545#tab=citedBy

U2 - 10.1109/CDC42340.2020.9304202

DO - 10.1109/CDC42340.2020.9304202

M3 - Conference contribution

AN - SCOPUS:85099880545

T3 - Proceedings of the IEEE Conference on Decision and Control

SP - 1902

EP - 1907

BT - 2020 59th IEEE Conference on Decision and Control, CDC 2020

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 59th IEEE Conference on Decision and Control, CDC 2020

Y2 - 14 December 2020 through 18 December 2020

ER -

Computing Stabilizing Linear Controllers via Policy Iteration

Abstract

Publication series

Conference

Bibliographical note

Publisher link

Find It

Other files and links

Fingerprint

Cite this