水利学报2026，Vol.57Issue(7)：985-1003,19.DOI:10.3724/j.slxb.20250559
水库群发电调度多智能体深度强化学习方法
Multi-agent deep reinforcement learning method for cascade reservoir operation for power generation
摘要
Abstract
In the optimization of cascade reservoir operation for power generation,traditional methods such as dynamic programming suffer from bottlenecks like low computational efficiency,limiting their practical engineering applications.Although single-agent deep reinforcement learning(DRL)can achieve continuous control in an end-to-end manner,it struggles to effectively capture the dynamic couplings among cascade reservoirs,potentially leading to unstable policy gradients and training oscillations.Moreover,treating the cascade reservoir system as a single entity limits scalability.To address these issues,this paper proposes a multi-agent deep reinforcement learning(MADRL)method for cascade reservoir operation based on the"Centralized Training and Decentralized Execution"framework.Each reservoir is treated as an independent agent,and a collaborative network integrating local decision-making with global information sharing is constructed to enhance training stability.A global reward function is designed,combin-ing power generation benefit with constraint penalty terms.Furthermore,the Copula-Gibbs joint distribution is intro-duced to generate runoff scenarios,improving adaptability to inflow uncertainty.Finally,hyperparameter combina-tions,including network architecture,discount factor,and learning rate,are tuned through sensitivity analysis and grid search.Engineering application results show that,under identical hardware and data conditions,the proposed method can achieve faster convergence.While strictly adhering to end-of-period water level and operational safety constraints,it achieves online inference times of 6.5 to 8.5 ms in 4 and 6 reservoirs cascaded system,which is approximately two orders of magnitude faster than discrete differential dynamic programming(DDDP).Under dry,normal,and wet typical inflow scenarios,the annual power generation deviation is controlled within 0.65%,and the end-of-period water level control accuracy is satisfactory.In conclusion,the multi-agent collaborative mechanism effectively addresses challenges posed by high-dimensional decision-making and system coupling,enhancing the environmental adaptability and engineering practicality of operational strategies.This provides reliable methodologi-cal support for efficient operation of large-scale cascade reservoirs.关键词
水库群调度/多智能体强化学习/深度神经网络/实时决策推理Key words
cascade reservoir operation/multi-agent reinforcement learning/deep neural networks/real-time decision-making分类
建筑与水利引用本文复制引用
冯仲恺,罗涛,关铁生,牛文静..水库群发电调度多智能体深度强化学习方法[J].水利学报,2026,57(7):985-1003,19.基金项目
国家自然科学基金项目(52394234,52441901,52379009) （52394234,52441901,52379009）
江苏省自然科学基金优秀青年基金项目(BK20240189) （BK20240189）
北京江河水利发展基金会—水利青年科技英才项目(JHYC202310) （JHYC202310）
江苏省科技智库计划项目(JSKX0225047) （JSKX0225047）
