A deep reinforcement learning algorithm for the power order optimization allocation of AGC in interconnected power grids

Lei Xi; Lipeng Zhou; Lang Liu; Dongliang Duan; Yanchun Xu; Liuqing Yang; Shouxiang Wang

doi:10.17775/CSEEJPES.2019.01840

A deep reinforcement learning algorithm for the power order optimization allocation of AGC in interconnected power grids

Lei Xi, Lipeng Zhou, Lang Liu, Dongliang Duan, Yanchun Xu, Liuqing Yang, Shouxiang Wang

Electrical and Computer Engineering

Research output: Contribution to journal › Article › peer-review

18 Scopus citations

Abstract

The integration of distributed generations (solar power, wind power), energy storage devices, and electric vehicles, causes unpredictable disturbances in power grids. It has become a top priority to coordinate the distributed generations, loads, and energy storages in order to better facilitate the utilization of new energy. Therefore, a novel algorithm based on deep reinforcement learning, namely the deep PDWoLF-PHC (policy dynamics based win or learn fast-policy hill climbing) network (DPDPN), is proposed to allocate power order among the various generators. The proposed algorithm combines the decision mechanism of reinforcement learning with the prediction mechanism of a deep neural network to obtain the optimal coordinated control for the source-grid-load. Consequently it solves the problem brought by stochastic disturbances and improves the utilization rate of new energy. Simulations are conducted with the case of the improved IEEE two-area and a case in the Guangdong power grid. Results show that the adaptability and control performance of the power system are improved using the proposed algorithm as compared with using other existing strategies.

Original language	English (US)
Article number	9056999
Pages (from-to)	712-723
Number of pages	12
Journal	CSEE Journal of Power and Energy Systems
Volume	6
Issue number	3
DOIs	https://doi.org/10.17775/CSEEJPES.2019.01840
State	Published - Sep 2020

Bibliographical note

Funding Information:
Manuscript received August 9, 2019; revised March 3, 2020; accepted March 12, 2020. Date of online publication April 6, 2020; date of current version May 20, 2020. This work was supported in part by the National Natural Science Foundation of China under Grant No. 51707102. L. Xi, L. P. Zhou, and Y. C. Xu are with the College of Electrical Engineering and New Energy, China Three Gorges University, Yichang 443002, China. L. Liu is with the State Grid Xianning Electric Power Supply Company, Xianning 437100, China. D. L. Duan is with the Department of Electrical and Computer Engineering, University of Wyoming, Laramie, WY, USA. L. Q. Yang is with the Department of Electrical and Computer Engineering, Colorado State University, Fort Collins, CO, USA. S. X. Wang (corresponding author, e-mail: sxwang@tju.edu.cn) is with the Key Laboratory of Smart Grid of Ministry of Education, Tianjin University, Tianjin 300072, China. DOI: 10.17775/CSEEJPES.2019.01840

Publisher Copyright:
© 2015 CSEE.

Keywords

Automatic generation control
DPDPN
deep reinforcement learning
power order allocation

Publisher link

10.17775/CSEEJPES.2019.01840

Find It

Find It at U of M Libraries

Cite this

@article{435cd781d37b4e7c96e0ff1b1e81b26f,

title = "A deep reinforcement learning algorithm for the power order optimization allocation of AGC in interconnected power grids",

abstract = "The integration of distributed generations (solar power, wind power), energy storage devices, and electric vehicles, causes unpredictable disturbances in power grids. It has become a top priority to coordinate the distributed generations, loads, and energy storages in order to better facilitate the utilization of new energy. Therefore, a novel algorithm based on deep reinforcement learning, namely the deep PDWoLF-PHC (policy dynamics based win or learn fast-policy hill climbing) network (DPDPN), is proposed to allocate power order among the various generators. The proposed algorithm combines the decision mechanism of reinforcement learning with the prediction mechanism of a deep neural network to obtain the optimal coordinated control for the source-grid-load. Consequently it solves the problem brought by stochastic disturbances and improves the utilization rate of new energy. Simulations are conducted with the case of the improved IEEE two-area and a case in the Guangdong power grid. Results show that the adaptability and control performance of the power system are improved using the proposed algorithm as compared with using other existing strategies. ",

keywords = "Automatic generation control, DPDPN, deep reinforcement learning, power order allocation",

author = "Lei Xi and Lipeng Zhou and Lang Liu and Dongliang Duan and Yanchun Xu and Liuqing Yang and Shouxiang Wang",

note = "Funding Information: Manuscript received August 9, 2019; revised March 3, 2020; accepted March 12, 2020. Date of online publication April 6, 2020; date of current version May 20, 2020. This work was supported in part by the National Natural Science Foundation of China under Grant No. 51707102. L. Xi, L. P. Zhou, and Y. C. Xu are with the College of Electrical Engineering and New Energy, China Three Gorges University, Yichang 443002, China. L. Liu is with the State Grid Xianning Electric Power Supply Company, Xianning 437100, China. D. L. Duan is with the Department of Electrical and Computer Engineering, University of Wyoming, Laramie, WY, USA. L. Q. Yang is with the Department of Electrical and Computer Engineering, Colorado State University, Fort Collins, CO, USA. S. X. Wang (corresponding author, e-mail: sxwang@tju.edu.cn) is with the Key Laboratory of Smart Grid of Ministry of Education, Tianjin University, Tianjin 300072, China. DOI: 10.17775/CSEEJPES.2019.01840 Publisher Copyright: {\textcopyright} 2015 CSEE.",

year = "2020",

month = sep,

doi = "10.17775/CSEEJPES.2019.01840",

language = "English (US)",

volume = "6",

pages = "712--723",

journal = "CSEE Journal of Power and Energy Systems",

issn = "2096-0042",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "3",

}

TY - JOUR

T1 - A deep reinforcement learning algorithm for the power order optimization allocation of AGC in interconnected power grids

AU - Xi, Lei

AU - Zhou, Lipeng

AU - Liu, Lang

AU - Duan, Dongliang

AU - Xu, Yanchun

AU - Yang, Liuqing

AU - Wang, Shouxiang

N1 - Funding Information: Manuscript received August 9, 2019; revised March 3, 2020; accepted March 12, 2020. Date of online publication April 6, 2020; date of current version May 20, 2020. This work was supported in part by the National Natural Science Foundation of China under Grant No. 51707102. L. Xi, L. P. Zhou, and Y. C. Xu are with the College of Electrical Engineering and New Energy, China Three Gorges University, Yichang 443002, China. L. Liu is with the State Grid Xianning Electric Power Supply Company, Xianning 437100, China. D. L. Duan is with the Department of Electrical and Computer Engineering, University of Wyoming, Laramie, WY, USA. L. Q. Yang is with the Department of Electrical and Computer Engineering, Colorado State University, Fort Collins, CO, USA. S. X. Wang (corresponding author, e-mail: sxwang@tju.edu.cn) is with the Key Laboratory of Smart Grid of Ministry of Education, Tianjin University, Tianjin 300072, China. DOI: 10.17775/CSEEJPES.2019.01840 Publisher Copyright: © 2015 CSEE.

PY - 2020/9

Y1 - 2020/9

N2 - The integration of distributed generations (solar power, wind power), energy storage devices, and electric vehicles, causes unpredictable disturbances in power grids. It has become a top priority to coordinate the distributed generations, loads, and energy storages in order to better facilitate the utilization of new energy. Therefore, a novel algorithm based on deep reinforcement learning, namely the deep PDWoLF-PHC (policy dynamics based win or learn fast-policy hill climbing) network (DPDPN), is proposed to allocate power order among the various generators. The proposed algorithm combines the decision mechanism of reinforcement learning with the prediction mechanism of a deep neural network to obtain the optimal coordinated control for the source-grid-load. Consequently it solves the problem brought by stochastic disturbances and improves the utilization rate of new energy. Simulations are conducted with the case of the improved IEEE two-area and a case in the Guangdong power grid. Results show that the adaptability and control performance of the power system are improved using the proposed algorithm as compared with using other existing strategies.

AB - The integration of distributed generations (solar power, wind power), energy storage devices, and electric vehicles, causes unpredictable disturbances in power grids. It has become a top priority to coordinate the distributed generations, loads, and energy storages in order to better facilitate the utilization of new energy. Therefore, a novel algorithm based on deep reinforcement learning, namely the deep PDWoLF-PHC (policy dynamics based win or learn fast-policy hill climbing) network (DPDPN), is proposed to allocate power order among the various generators. The proposed algorithm combines the decision mechanism of reinforcement learning with the prediction mechanism of a deep neural network to obtain the optimal coordinated control for the source-grid-load. Consequently it solves the problem brought by stochastic disturbances and improves the utilization rate of new energy. Simulations are conducted with the case of the improved IEEE two-area and a case in the Guangdong power grid. Results show that the adaptability and control performance of the power system are improved using the proposed algorithm as compared with using other existing strategies.

KW - Automatic generation control

KW - DPDPN

KW - deep reinforcement learning

KW - power order allocation

UR - http://www.scopus.com/inward/record.url?scp=85091629273&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85091629273&partnerID=8YFLogxK

U2 - 10.17775/CSEEJPES.2019.01840

DO - 10.17775/CSEEJPES.2019.01840

M3 - Article

AN - SCOPUS:85091629273

SN - 2096-0042

VL - 6

SP - 712

EP - 723

JO - CSEE Journal of Power and Energy Systems

JF - CSEE Journal of Power and Energy Systems

IS - 3

M1 - 9056999

ER -

A deep reinforcement learning algorithm for the power order optimization allocation of AGC in interconnected power grids

Abstract

Bibliographical note

Keywords

Publisher link

Find It

Other files and links

Fingerprint

Cite this