TY - GEN
T1 - Recovering robustness in model-free reinforcement learning
AU - Venkataraman, Harish K.
AU - Seiler, Peter J.
N1 - Publisher Copyright:
© 2019 American Automatic Control Council.
Copyright:
Copyright 2020 Elsevier B.V., All rights reserved.
PY - 2019/7
Y1 - 2019/7
N2 - Reinforcement learning (RL) is used to directly design a control policy using data collected from the system. This paper considers the robustness of controllers trained via model-free RL. The discussion focuses on posing the (model-free) linear quadratic Gaussian (LQG) problem as a special instance of RL. A simple LQG example is used to demonstrate that RL with partial observations can lead to poor robustness margins. It is proposed to recover robustness by introducing random perturbations at the system input during the RL training. The perturbation magnitude can be used to trade off performance for increased robustness. Two simple examples are presented to demonstrate the proposed method for enhancing robustness during RL training.
AB - Reinforcement learning (RL) is used to directly design a control policy using data collected from the system. This paper considers the robustness of controllers trained via model-free RL. The discussion focuses on posing the (model-free) linear quadratic Gaussian (LQG) problem as a special instance of RL. A simple LQG example is used to demonstrate that RL with partial observations can lead to poor robustness margins. It is proposed to recover robustness by introducing random perturbations at the system input during the RL training. The perturbation magnitude can be used to trade off performance for increased robustness. Two simple examples are presented to demonstrate the proposed method for enhancing robustness during RL training.
UR - http://www.scopus.com/inward/record.url?scp=85072295039&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85072295039&partnerID=8YFLogxK
U2 - 10.23919/acc.2019.8815368
DO - 10.23919/acc.2019.8815368
M3 - Conference contribution
AN - SCOPUS:85072295039
T3 - Proceedings of the American Control Conference
SP - 4210
EP - 4216
BT - 2019 American Control Conference, ACC 2019
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2019 American Control Conference, ACC 2019
Y2 - 10 July 2019 through 12 July 2019
ER -