Two-timescale voltage regulation in distribution grids using deep reinforcement learning

Qiuling Yang, Gang Wang, Alireza Sadeghi, Georgios B. Giannakis, Jian Sun

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Frequent and sizeable voltage fluctuations become more pronounced with the increasing penetration of distributed renewable generation, and they considerably challenge distribution grids. Voltage regulation schemes so far have relied on either utility-owned devices (e.g., voltage transformers, and shunt capacitors), or more recently, smart power inverters that come with contemporary distributed generation units (e.g., photovoltaic systems, and wind turbines). Nonetheless, due to the distinct response times of those devices, as well as the discrete on-off commitment of capacitor units, joint control of both types of assets is challenging. In this context, a novel two-timescale voltage regulation scheme is developed here by coupling optimization with reinforcement learning advances. Shunt capacitors are configured on a slow timescale (e.g., daily basis) leveraging a deep reinforcement learning algorithm, while optimal setpoints of the power inverters are computed using a linearized distribution flow model on a fast timescale (e.g., every few seconds or minutes). Numerical experiments using a real-world 47-bus distribution feeder showcase the remarkable performance of the novel scheme.

Original languageEnglish (US)
Title of host publication2019 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids, SmartGridComm 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781538680995
DOIs
StatePublished - Oct 2019
Event2019 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids, SmartGridComm 2019 - Beijing, China
Duration: Oct 21 2019Oct 23 2019

Publication series

Name2019 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids, SmartGridComm 2019

Conference

Conference2019 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids, SmartGridComm 2019
CountryChina
CityBeijing
Period10/21/1910/23/19

Fingerprint

Reinforcement learning
Reinforcement Learning
reinforcement
Voltage control
Time Scales
Capacitors
Capacitor
Voltage
Grid
regulation
learning
Distributed power generation
Electric potential
Distributed Generation
Photovoltaic System
Wind turbines
Learning algorithms
fluctuation
Unit
Wind Turbine

Keywords

  • Capacitor
  • Deep reinforcement learning.
  • Inverter
  • Two-timescale
  • Voltage regulation

Cite this

Yang, Q., Wang, G., Sadeghi, A., Giannakis, G. B., & Sun, J. (2019). Two-timescale voltage regulation in distribution grids using deep reinforcement learning. In 2019 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids, SmartGridComm 2019 [8909764] (2019 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids, SmartGridComm 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SmartGridComm.2019.8909764

Two-timescale voltage regulation in distribution grids using deep reinforcement learning. / Yang, Qiuling; Wang, Gang; Sadeghi, Alireza; Giannakis, Georgios B.; Sun, Jian.

2019 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids, SmartGridComm 2019. Institute of Electrical and Electronics Engineers Inc., 2019. 8909764 (2019 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids, SmartGridComm 2019).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Yang, Q, Wang, G, Sadeghi, A, Giannakis, GB & Sun, J 2019, Two-timescale voltage regulation in distribution grids using deep reinforcement learning. in 2019 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids, SmartGridComm 2019., 8909764, 2019 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids, SmartGridComm 2019, Institute of Electrical and Electronics Engineers Inc., 2019 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids, SmartGridComm 2019, Beijing, China, 10/21/19. https://doi.org/10.1109/SmartGridComm.2019.8909764
Yang Q, Wang G, Sadeghi A, Giannakis GB, Sun J. Two-timescale voltage regulation in distribution grids using deep reinforcement learning. In 2019 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids, SmartGridComm 2019. Institute of Electrical and Electronics Engineers Inc. 2019. 8909764. (2019 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids, SmartGridComm 2019). https://doi.org/10.1109/SmartGridComm.2019.8909764
Yang, Qiuling ; Wang, Gang ; Sadeghi, Alireza ; Giannakis, Georgios B. ; Sun, Jian. / Two-timescale voltage regulation in distribution grids using deep reinforcement learning. 2019 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids, SmartGridComm 2019. Institute of Electrical and Electronics Engineers Inc., 2019. (2019 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids, SmartGridComm 2019).
@inproceedings{dd58235bc3e046988987916036ba8a44,
title = "Two-timescale voltage regulation in distribution grids using deep reinforcement learning",
abstract = "Frequent and sizeable voltage fluctuations become more pronounced with the increasing penetration of distributed renewable generation, and they considerably challenge distribution grids. Voltage regulation schemes so far have relied on either utility-owned devices (e.g., voltage transformers, and shunt capacitors), or more recently, smart power inverters that come with contemporary distributed generation units (e.g., photovoltaic systems, and wind turbines). Nonetheless, due to the distinct response times of those devices, as well as the discrete on-off commitment of capacitor units, joint control of both types of assets is challenging. In this context, a novel two-timescale voltage regulation scheme is developed here by coupling optimization with reinforcement learning advances. Shunt capacitors are configured on a slow timescale (e.g., daily basis) leveraging a deep reinforcement learning algorithm, while optimal setpoints of the power inverters are computed using a linearized distribution flow model on a fast timescale (e.g., every few seconds or minutes). Numerical experiments using a real-world 47-bus distribution feeder showcase the remarkable performance of the novel scheme.",
keywords = "Capacitor, Deep reinforcement learning., Inverter, Two-timescale, Voltage regulation",
author = "Qiuling Yang and Gang Wang and Alireza Sadeghi and Giannakis, {Georgios B.} and Jian Sun",
year = "2019",
month = "10",
doi = "10.1109/SmartGridComm.2019.8909764",
language = "English (US)",
series = "2019 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids, SmartGridComm 2019",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
booktitle = "2019 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids, SmartGridComm 2019",

}

TY - GEN

T1 - Two-timescale voltage regulation in distribution grids using deep reinforcement learning

AU - Yang, Qiuling

AU - Wang, Gang

AU - Sadeghi, Alireza

AU - Giannakis, Georgios B.

AU - Sun, Jian

PY - 2019/10

Y1 - 2019/10

N2 - Frequent and sizeable voltage fluctuations become more pronounced with the increasing penetration of distributed renewable generation, and they considerably challenge distribution grids. Voltage regulation schemes so far have relied on either utility-owned devices (e.g., voltage transformers, and shunt capacitors), or more recently, smart power inverters that come with contemporary distributed generation units (e.g., photovoltaic systems, and wind turbines). Nonetheless, due to the distinct response times of those devices, as well as the discrete on-off commitment of capacitor units, joint control of both types of assets is challenging. In this context, a novel two-timescale voltage regulation scheme is developed here by coupling optimization with reinforcement learning advances. Shunt capacitors are configured on a slow timescale (e.g., daily basis) leveraging a deep reinforcement learning algorithm, while optimal setpoints of the power inverters are computed using a linearized distribution flow model on a fast timescale (e.g., every few seconds or minutes). Numerical experiments using a real-world 47-bus distribution feeder showcase the remarkable performance of the novel scheme.

AB - Frequent and sizeable voltage fluctuations become more pronounced with the increasing penetration of distributed renewable generation, and they considerably challenge distribution grids. Voltage regulation schemes so far have relied on either utility-owned devices (e.g., voltage transformers, and shunt capacitors), or more recently, smart power inverters that come with contemporary distributed generation units (e.g., photovoltaic systems, and wind turbines). Nonetheless, due to the distinct response times of those devices, as well as the discrete on-off commitment of capacitor units, joint control of both types of assets is challenging. In this context, a novel two-timescale voltage regulation scheme is developed here by coupling optimization with reinforcement learning advances. Shunt capacitors are configured on a slow timescale (e.g., daily basis) leveraging a deep reinforcement learning algorithm, while optimal setpoints of the power inverters are computed using a linearized distribution flow model on a fast timescale (e.g., every few seconds or minutes). Numerical experiments using a real-world 47-bus distribution feeder showcase the remarkable performance of the novel scheme.

KW - Capacitor

KW - Deep reinforcement learning.

KW - Inverter

KW - Two-timescale

KW - Voltage regulation

UR - http://www.scopus.com/inward/record.url?scp=85076436632&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85076436632&partnerID=8YFLogxK

U2 - 10.1109/SmartGridComm.2019.8909764

DO - 10.1109/SmartGridComm.2019.8909764

M3 - Conference contribution

AN - SCOPUS:85076436632

T3 - 2019 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids, SmartGridComm 2019

BT - 2019 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids, SmartGridComm 2019

PB - Institute of Electrical and Electronics Engineers Inc.

ER -