Two-Timescale Voltage Control in Distribution Grids Using Deep Reinforcement Learning

Qiuling Yang, Gang Wang, Alireza Sadeghi, Georgios B. Giannakis, Jian Sun

Research output: Contribution to journalArticlepeer-review

10 Scopus citations

Abstract

Modern distribution grids are currently being challenged by frequent and sizable voltage fluctuations, due mainly to the increasing deployment of electric vehicles and renewable generators. Existing approaches to maintaining bus voltage magnitudes within the desired region can cope with either traditional utility-owned devices (e.g., shunt capacitors), or contemporary smart inverters that come with distributed generation units (e.g., photovoltaic plants). The discrete on-off commitment of capacitor units is often configured on an hourly or daily basis, yet smart inverters can be controlled within milliseconds, thus challenging joint control of these two types of assets. In this context, a novel two-timescale voltage regulation scheme is developed for distribution grids by judiciously coupling data-driven with physics-based optimization. On a faster timescale, say every second, the optimal setpoints of smart inverters are obtained by minimizing instantaneous bus voltage deviations from their nominal values, based on either the exact alternating current power flow model or a linear approximant of it; whereas, on the slower timescale (e.g., every hour), shunt capacitors are configured to minimize the long-term discounted voltage deviations using a deep reinforcement learning algorithm. Extensive numerical tests on a real-world 47-bus distribution network as well as the IEEE 123-bus test feeder using real data corroborate the effectiveness of the novel scheme.

Original languageEnglish (US)
Article number8892476
Pages (from-to)2313-2323
Number of pages11
JournalIEEE Transactions on Smart Grid
Volume11
Issue number3
DOIs
StatePublished - May 2020

Bibliographical note

Funding Information:
Manuscript received April 19, 2019; revised July 23, 2019 and September 4, 2019; accepted October 30, 2019. Date of publication November 6, 2019; date of current version April 21, 2020. The work of Q. Yang was supported in part by the National Natural Science Foundation of China under Grant 61522303, Grant 61720106011, and Grant 61621063, and in part by the China Scholarship Council. The work of G. Wang, A. Sadeghi, and G. B. Giannakis was supported by the National Science Foundation under Grant 1509040, Grant 1711471, and Grant 1901134. The work of J. Sun was supported in part by the National Natural Science Foundation of China under Grant 61522303, Grant 61720106011, and Grant 61621063. Paper no. TSG-00578-2019. (Corresponding author: Jian Sun.) Q. Yang and J. Sun are with the State Key Laboratory of Intelligent Control and Decision of Complex Systems, School of Automation, Beijing Institute of Technology, Beijing 100081, China (e-mail: yang6726@umn.edu; sunjian@bit.edu.cn).

Keywords

  • Two timescales
  • capacitors
  • deep reinforcement learning
  • inverters
  • voltage control

Fingerprint Dive into the research topics of 'Two-Timescale Voltage Control in Distribution Grids Using Deep Reinforcement Learning'. Together they form a unique fingerprint.

Cite this