电工技术学报2025，Vol.40Issue(5)：1487-1502,16.DOI:10.19595/j.cnki.1000-6753.tces.240316
基于深度强化学习的有源配电网多时间尺度源荷储协同优化调控
Multi-Timescale Optimal Dispatch of Source-Load-Storage Coordination in Active Distribution Network Based on Deep Reinforcement Learning
摘要
Abstract
The stochastic,volatile,and uncontrollable nature of renewable energy generation such as wind and solar leads to increased uncertainty and difficulty in regulation with the expansion of installed capacity.Consequently,the safe,economical,and stable operation of distribution networks faces formidable challenges.The data-driven artificial intelligence method has the characteristics of model-free and self-adaptation.It can independently learn the complex uncertainty of wind and solar energy and load,and has a good supporting role for the optimal regulation of active distribution network.Consider the problem of high dimensional variable information redundancy in day-ahead scheduling and the dilemma of stabilizing fluctuating power and tracking day ahead scheduling in intraday scheduling,this paper proposes a multi-timescale intelligent optimal regulation method for active distribution network based on deep reinforcement learning algorithm. Firstly,a multi time scale framework for active distribution networks(AND)was constructed based on the operational characteristics and regulatory requirements of the equipment,and the corresponding state inputs and action outputs of deep reinforcement learning(DRL)agents at different time scales were described.In the day-ahead stage,this paper takes the lowest daily operation cost of ADN and the smoothest tie line power as the optimization objective.In order to reduce the influence of high-dimensional variable information redundancy in the day ahead state space on the optimization effect,an improved feature extraction method considering the spatio-temporal coupling characteristics of the day-ahead state information is proposed.In the intraday stage,considering the full absorption of uncertain power will cause deviations from the pre-scheduled operating status of controllable equipment as energy storage system(ESS),this paper designs a reward function that reasonably reflects the tie line power deviation and state of charge(SOC)deviation of ESS,and realizes the optimization in the whole scheduling period through the deep reinforcement learning method. Simulation results show that agents on both time scales can converge effectively.In the day ahead stage,while controlling the daily comprehensive operation cost of ADN,the peak value on the tie line is smoothed.The active power loss and the mean square deviation of tie line power is reduced.Compared with using fully connected network and n×n convolutional network as feature extraction network,the proposed feature extraction method has improved the convergence and convergence speed of DRL agent.In the intraday stage,the proposed method realizes the adaptive stabilization of fluctuating power while tracking the day ahead plan of scheduling equipment.The optimization effect of the proposed method in the whole scheduling period is better than that of the full consumption and single-step optimization. The following conclusions can be drawn from the simulation analysis:(1)Using the data-driven deep reinforcement learning method,the probability information of uncertainty in the data set can be learned autonomously.The feature extraction network designed according to the spatio-temporal coupling characteristics of multi period state space variables can effectively improve the optimization effect and convergence of the deep learning method.(2)Without assuming the probability distribution of source and load uncertainty,the day-ahead dispatch realizes the multi-objective optimization of the ADN with the lowest daily operation cost and the smoothest tie line power.(3)The intraday dispatch takes into account the offset between the actual power and the day ahead forecast power of the source and load,and realizes the real-time stabilization of the fluctuation of uncertain power while tracking the day-ahead plan.关键词
有源配电网/优化调控/源荷储协同/深度强化学习Key words
Active distribution network/optimal dispatch/source-load-storage coordination/deep reinforcement learning分类
信息技术与安全科学引用本文复制引用
李鹏,钟瀚明,马红伟,李建锋,刘洋,王加浩..基于深度强化学习的有源配电网多时间尺度源荷储协同优化调控[J].电工技术学报,2025,40(5):1487-1502,16.基金项目
国家电网公司科技项目资助(4000-202316071A-1-1-ZN). （4000-202316071A-1-1-ZN）
