计算机工程与科学2025，Vol.47Issue(3)：422-433,12.DOI:10.3969/j.issn.1007-130X.2025.03.005
数据中心制冷系统强化学习控制
Reinforcement learning control for data center refrigeration systems
摘要
Abstract
The refrigeration system in data centers needs to operate continuously throughout the year,and its energy consumption cannot be ignored.Moreover,traditional PID control methods strug-gle to achieve overall energy savings for the system.To address this,a reinforcement learning control strategy is proposed for data center refrigeration systems,with the control objective of enhancing the overall energy efficiency of the system while meeting cooling requirements.A two-layer hierarchical con-trol structure is designed.The upper optimization layer introduces the multistep prediction-deep deter-ministic policy gradient(MP-DDPG)algorithm,which leverages DDPG to handle the multi-dimensional continuous action space of the refrigeration system to determine the water valve opening of the air hand-ling unit and the optimal setpoint for each loop in the chilling station system.Multistep prediction is em-ployed to enhance algorithm efficiency and overcome the impact of large system delay during real-time control.The lower field control layer uses PID control to enable the controlled variables to track the op-timal setpoints obtained from the optimization layer,achieving performance optimization without dis-rupting the existing field control system.To address the challenge of real-time control with model-free reinforcement learning,a system prediction model is first constructed,and the reinforcement learning controller is trained offline through interaction with this model.Subsequently,online real-time control is implemented.Experimental results show that compared to the traditional DDPG algorithm,the learn-ing efficiency of the controller is improved by 50％.Compared to PID and MP-DQN(multistep prediction-deep Q network),the system's dynamic performance is improved,and the whole energy effi-ciency is increased by approximately 30.149％and 11.6％,respectively.关键词
数据中心制冷系统/预测控制/强化学习/深度确定性策略梯度法/集成学习Key words
data center refrigeration system/predictive control/reinforcement learning/depth deter-ministic strategy gradient method/integrated learning分类
计算机与自动化引用本文复制引用
魏东,贾宇辰,韩少然..数据中心制冷系统强化学习控制[J].计算机工程与科学,2025,47(3):422-433,12.基金项目
国家自然科学基金(62371032) （62371032）
北京市自然科学基金(4232021) （4232021）
住房城乡建设部科学技术项目(研究开发项目)(2019-K-149) （研究开发项目）
北京建筑大学高级主讲教师培育计划(GJZJ20220803) （GJZJ20220803）
