电力工程技术2026，Vol.45Issue(5)：69-80,12.DOI:10.12158/j.2096-3203.2026.05.007
基于多智能体深度强化学习的多区域负荷频率协同控制
Multi-area load frequency cooperative control based on multi-agent deep reinforcement learning
摘要
Abstract
With the integration of large-scale renewable energy,the frequency stability of power systems has been subjected to severe challenges.In this study,an expert-prefilled multi-agent twin delayed deep deterministic policy gradient(EP-MATD3)algorithm based on an expert data pre-filling mechanism is proposed for multi-area load frequency control.Firstly,a multi-area frequency response model including thermal power units,wind turbines,photovoltaic systems,and energy storage systems is first established.Based on the traditional multi-area tie-line power model,coordinated control between regional controllers is incorporated,by which the interconnection between regions is strengthened and unplanned power exchanges are reduced.Then,the multi-agent twin delayed deep deterministic policy gradient(MATD3)algorithm is adopted to mitigate the Q-value overestimation problem inherent in traditional reinforcement learning,and stability and convergence of the control policy are enhanced.Furthermore,within the collaborative control framework of centralized training and decentralized execution,an expert data pre-filling mechanism is introduced during the centralized training stage,whereby the occurrence of invalid actions during random exploration is limited and the convergence of agent training is accelerated.During the decentralized execution stage,unit power outputs are independently adjusted by the trained agents according to the real-time system states of their respective regions,enabling effective suppression of frequency fluctuations.Through simulation on a three-area power system,it is demonstrated that,compared with traditional methods,the proposed EP-MATD3 control strategy achieves a significant reduction in training time and effectively decreases system frequency deviations under continuous step-load and photovoltaic fluctuation disturbances,thereby verifying its effectiveness and superiority in the frequency control of complex power systems.关键词
多区域负荷频率控制/协同控制/区域控制误差/多智能体深度强化学习/双延迟深度确定性策略梯度算法/预填充机制Key words
multi-area load frequency control/cooperative control/area control error/multi-agent deep reinforcement learning/twin delay deep deterministic policy gradient algorithm/pre-filling mechanism分类
信息技术与安全科学引用本文复制引用
许庆禹,何宇,张靖,沈涛,齐岳,赵健..基于多智能体深度强化学习的多区域负荷频率协同控制[J].电力工程技术,2026,45(5):69-80,12.基金项目
国家自然科学基金资助项目(52406227) （52406227）
贵州省科技支撑项目(黔科合成果LH[2025]重点014) （黔科合成果LH[2025]重点014）
