电工技术学报2026，Vol.41Issue(11)：3742-3754,13.DOI:10.19595/j.cnki.1000-6753.tces.250960
基于延迟感知多智能体深度强化学习的多光储直柔系统优化调度
Optimized Scheduling of Multi Photovoltaics and Energy Storage Integrated Flexible Direct Current Distribution Systems Based on Delay-Aware Multi-Agent Deep Reinforcement Learning
摘要
Abstract
In the context of developing a new power system dominated by new energy,large-scale integration of distributed photovoltaics has exacerbated spatiotemporal mismatches.DC-AC conversion losses in prevalent AC distribution systems have also grown more prominent.These issues drove rapid advancement of the photovoltaic,energy storage,direct current,flexibility(PEDF)system.Most existing studies,however,focus on single-system optimization and neglect collaborative scheduling in multiple photovoltaic,energy storage,direct current,flexibility(MPEDF)systems.To enhance MPEDF power complementarity and cut operational costs,this study proposed a collaborative optimal scheduling strategy based on multi-agent deep reinforcement learning. First,the study established an interconnected operation framework for the MPEDF system.Multiple independent PEDF systems integrated PV,grid access,battery energy storage(BES),electric vehicles(EVs),and flexible loads via DC-DC converters and interconnection lines.The power response of flexible loads,BES,and EVs correlates linearly with DC bus voltage deviation.EV charging behavior was simulated using the Monte Carlo method.Next,the study developed a power control method.It dynamically guides the charging and discharging power of flexible devices and EVs based on DC bus voltage deviation signals,enabling power scheduling.By optimizing the voltage gain coefficient at each scheduling moment,the study obtained a power adjustment plan for flexible devices in PEDF systems to achieve economic operation.Finally,the study constructed a complete MPEDF system optimization scheduling model with a power interaction strategy.Its power transmission mechanism coordinates inter-system power differences to minimize operational costs.The model considers various costs,including power transmission,device scheduling,and curtailment penalties,aiming to maximize economic and environmental benefits. To address data privacy and system-specific operational traits,the study employed multi-agent deep reinforcement learning for model solving.For information interaction delays among agents,it adopted a delay-aware multi-agent proximal policy optimization(DA-MAPPO)algorithm.The algorithm introduces a delay-aware Markov process and expands the state space,effectively reducing interaction delays and boosting solving efficiency.Simulations on three interconnected MPEDF systems show that enhanced power complementarity increases photovoltaic absorption and cuts operating costs.Compared with scenarios without electric vehicle participation or inter-system power interaction,electricity purchase costs drop by 42.7％and curtailment decreases by 38.5％.DA-MAPPO yields a total operating cost of 2 640.21 yuan,17.3％lower than CPLEX,13.2％lower than PSO,9.9％lower than SAC,and 6.1％lower than MAPPO.It also has the fastest solving speed(1 second),making it suitable for real-time scheduling. In conclusion,this study develops an operation framework for the PEDF system based on DC bus voltage control strategies.This framework adjusts the power of flexible devices and facilitates inter-system power interaction through voltage signals.The strategy contributes to balancing renewable energy supply and demand,supporting the stable and low-carbon operation of urban power grids,and provides new insights for future optimization scheduling models.Future research will explore the integration of MPEDF with shared energy storage to address spatial and temporal differences in PV output and load demand,as well as to solve issues of energy storage redundancy in some systems and capacity insufficiency in others.关键词
多光储直柔系统/深度强化学习/功率交互/优化调度/延迟感知多智能体Key words
Multiple photovoltaic,energy storage,direct current,flexibility(MPEDF)system/deep reinforcement learning/power interaction/optimization scheduling/delay-aware multi-agent分类
信息技术与安全科学引用本文复制引用
彭春华,张浩旗,孙惠娟,卢恒宇..基于延迟感知多智能体深度强化学习的多光储直柔系统优化调度[J].电工技术学报,2026,41(11):3742-3754,13.基金项目
国家自然科学基金(52267007,52167009,52567008)和江西省自然科学基金(20242BAB26070)资助项目. （52267007,52167009,52567008）
