Recovering robustness in model-free reinforcement learning

Harish K. Venkataraman, Peter J. Seiler

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Reinforcement learning (RL) is used to directly design a control policy using data collected from the system. This paper considers the robustness of controllers trained via model-free RL. The discussion focuses on posing the (model-free) linear quadratic Gaussian (LQG) problem as a special instance of RL. A simple LQG example is used to demonstrate that RL with partial observations can lead to poor robustness margins. It is proposed to recover robustness by introducing random perturbations at the system input during the RL training. The perturbation magnitude can be used to trade off performance for increased robustness. Two simple examples are presented to demonstrate the proposed method for enhancing robustness during RL training.

Original languageEnglish (US)
Title of host publication2019 American Control Conference, ACC 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages4210-4216
Number of pages7
ISBN (Electronic)9781538679265
StatePublished - Jul 2019
Event2019 American Control Conference, ACC 2019 - Philadelphia, United States
Duration: Jul 10 2019Jul 12 2019

Publication series

NameProceedings of the American Control Conference
Volume2019-July
ISSN (Print)0743-1619

Conference

Conference2019 American Control Conference, ACC 2019
CountryUnited States
CityPhiladelphia
Period7/10/197/12/19

    Fingerprint

Cite this

Venkataraman, H. K., & Seiler, P. J. (2019). Recovering robustness in model-free reinforcement learning. In 2019 American Control Conference, ACC 2019 (pp. 4210-4216). [8815368] (Proceedings of the American Control Conference; Vol. 2019-July). Institute of Electrical and Electronics Engineers Inc..