Risk-aware Energy Management of Extended Range Electric Delivery Vehicles with Implicit Quantile Network

Pengyue Wang, Yan Li, Shashi Shekhar, William F. Northrop

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Model-free reinforcement learning (RL) algorithms are used to solve sequential decision-making problems under uncertainty. They are data-driven methods and do not require an explicit model of the studied system or environment. Because of this characteristic, they are widely utilized in Intelligent Transportation Systems (ITS), as real-world transportation systems are highly complex and extremely difficult to model. However, in most literature, decisions are made according to the expected long-term return estimated by the RL algorithm, ignoring the underlying risk. In this work, a distributional RL algorithm called implicit quantile network is adapted for the energy management problem of a delivery vehicle. Instead of only estimating the expected long-term return, the full return distribution is estimated implicitly. This is highly beneficial for applications in ITS, as uncertainty and randomness are intrinsic characteristics of transportation systems. In addition, risk-aware strategies are integrated into the algorithm with the risk measure of conditional value at risk. In this study, we demonstrate that by changing a hyperparameter, the trade-off between fuel efficiency and the risk of running out of battery power during a delivery trip can be controlled according to different application scenarios and personal preferences.

Original languageEnglish (US)
Title of host publication2020 IEEE 16th International Conference on Automation Science and Engineering, CASE 2020
PublisherIEEE Computer Society
Pages772-778
Number of pages7
ISBN (Electronic)9781728169040
DOIs
StatePublished - Aug 2020
Event16th IEEE International Conference on Automation Science and Engineering, CASE 2020 - Hong Kong, Hong Kong
Duration: Aug 20 2020Aug 21 2020

Publication series

NameIEEE International Conference on Automation Science and Engineering
Volume2020-August
ISSN (Print)2161-8070
ISSN (Electronic)2161-8089

Conference

Conference16th IEEE International Conference on Automation Science and Engineering, CASE 2020
CountryHong Kong
CityHong Kong
Period8/20/208/21/20

Bibliographical note

Funding Information:
*Corresponding author The information, data, or work presented herein was funded in part by the Advanced Research Projects Agency-Energy (ARPA-E) U.S. Department of Energy, under Award Number DE-AR0000795.

Funding Information:
The information, data, or work presented herein was funded in part by the Advanced Research Projects Agency-Energy (ARPA-E) U.S. Department of Energy, under Award Number DE-AR0000795. The views and opinions of authors expressed herein do not necessarily state or reflect those of thenUitedtSteasoGvernment or anygaencyetrehof.

Publisher Copyright:
© 2020 IEEE.

Fingerprint Dive into the research topics of 'Risk-aware Energy Management of Extended Range Electric Delivery Vehicles with Implicit Quantile Network'. Together they form a unique fingerprint.

Cite this