电工技术学报2024，Vol.39Issue(19)：6054-6071,6103,19.DOI:10.19595/j.cnki.1000-6753.tces.231320
计及高渗透率光伏消纳与深度强化学习的综合能源系统预测调控
The Predictive-Control Optimization Method for Park Integrated Energy System Considering the High Penetration of Photovoltaics and Deep Reinforcement Learning
摘要
Abstract
As the interface between different energy infrastructures and users,park integrated energy system(PIES)has gained universal recognition for improving the reliability,resiliency,and profitability of multi-carrier energy systems by adaptively scheduling fast energy conversion units(e.g.,combined heat and power(CHP),gas boiler(GB),and electric boiler(EB))and participating in the various energy markets(e.g.,electricity,heat,and natural gas).As a promising technology for replacing the rule-based decision-making in PIES,deep reinforcement learning(DRL)is a practical solution to identify the optimal control for energy conversion equipment.However,as PIES's customers perform more casual energy-consumption behaviors,the intermittency and volatility of demands make managing multi-energy supply and storage much harder for DRL agents.To tackle this task,focusing on the utilization of high penetration photovoltaic and the optimization of PIES's benefits,this article proposes an optimization scheduling method for PIES that combines the deep reinforcement learning and the interval prediction of photovoltaic power generation,considering the uncertainty of photovoltaic power generation. Firstly,taking the equipment of energy conversion and storage as the scheduling objects,we design the predictive-control optimization structure,which can be divided into the facility level and information level,of PIES with electricity,gas,and heat,introducing the coordination between different sub-models.Secondly,the continuous and discrete feature data are respectively normalized and encoded for deterministic and probabilistic predicting the photovoltaic power generation based on temporal convolutional networks and kernel density estimation.Thirdly,based on the theory of model predictive control,the iteratively obtained intervals of photovoltaic power generation are used to construct the operating environment state of the control agent of soft actor critic(SAC)and to obtain the scheduling actions for PIES's equipment of energy conversion and storage. Numerical results show that the proposed PFP-SAC method is able to identify the generation of photovoltaic power,improve the utilization of PV generation,and optimize the benefit of PIES by dynamic scheduling these conversion and storage equipment and increasing their operation efficiency.Meanwhile,these results prove that the gaps of energy purchasing price is the motivation of multi-energy conversion for PIES and its cost-saving.On the contrast,in the scenario of high penetration of photovoltaic power,the multi-energy conversion and storage of PIES need to simultaneously consider the consumption demand for photovoltaic power and the price-gaps of multi energy,and improve its utilization of photovoltaic power generation as much as possible by reserving energy storage resources.Finally,taking the traditional SAC and deep deterministic policy gradient(DDPG)as the benchmarks,the same datasets are utilized to verify the performance of proposed method and benchmarks,including the scheduling benefit and SOC of storage.The results show that our proposed method is superior for each index. The following conclusions can be drawn from the simulation analysis:(1)A PIES model with multiple kinds of energy conversion and storage units are constructed,accompanying the uncertainty of renewable generation,demands,and energy purchasing prices.In this sense,it is closer to reality than existing PIES models.(2)Model predictive control theory and deep reinforcement learning algorithm are employed to cope with the intermittent nature of multi-energy demands.This paper constructs the state space of DRL models with prediction intervals of multi-energy demands of PIES,which is obtained by TCN and KDE.(3)Taking the operating cost saving as the prioritize objective and the generation utilization of photovoltaic power as secondary goal of PIES scheduling,soft actor critic,which is a promising DRL algorithm,is applied to reduce the operational expenditures and improve the usage of multi-energy storage capacity as much as possible.Compared with traditional DRL algorithms,it owns the advantages of predicting accuracy and the economic benefits of PIES management.关键词
综合能源系统/深度强化学习/柔性"行动器-判别器"/时序卷积网络/模型预测控制Key words
Integrated energy system/deep reinforcement learning/soft actor-critic/temporal convolutional network/model predictive control分类
信息技术与安全科学引用本文复制引用
陈明昊,朱月瑶,孙毅,谢志远,吴鹏..计及高渗透率光伏消纳与深度强化学习的综合能源系统预测调控[J].电工技术学报,2024,39(19):6054-6071,6103,19.基金项目
国家电网有限公司科技项目资助(52130X230008). （52130X230008）
