交通信息与安全2025，Vol.43Issue(6)：86-97,12.DOI:10.3963/j.jssn.1674-4861.2025.06.009
面向主线匝道协调的深度强化学习匝道控制方法
A Method of Deep Reinforcement Learning-based Ramp Metering for Mainline-ramp Coordination
摘要
Abstract
The merging area of expressway ramps is prone to traffic congestion and frequent accidents.To improve the performance of traditional ramp metering algorithms in terms of response speed and control accuracy,a ramp metering method based on reinforcement learning is studied.The ramp metering problem is formulated as a Markov decision process.The action space is designed using discrete signal phases to improve training efficiency.A state space and a multi-dimensional reward function are constructed to represent the operating states of the mainline and ramps.At the state perception level,a real time traffic detection mechanism is incorporated.To avoid high frequen-cy phase switching,a minimum phase duration constraint is imposed on action outputs.Meanwhile,prioritized expe-rience replay is used during the training process to enhance the model performance.Furthermore,the deep network structure is optimized to improve convergence speed and generalization in complex traffic environments.Residual connections and layer normalization are introduced to construct a lightweight and efficient multi-layer perception network.A microscopic simulation platform is used to conduct systematic experiments to verify the control effect of the proposed method.The results show that compared with the no-control scenario,the system throughput increased by 52.67%under the proposed mainline-ramp coordinated control.Meanwhile,the average travel time decreases by 58.21%under the proposed method.Moreover,traffic efficiency on the mainline and ramps improves significantly under the proposed method.The proposed method is deployed in the entrance traffic limiting project of the section from Hangzhou West to Yuqian Interchange on the Hangzhou-Huizhou Expressway.The road network structure and traffic flow characteristics of this section are accurately reproduced.The results indicate that network vehicle num-bers and mainline average speed increase,while speed fluctuation is more moderate.These improvements demon-strate that the proposed method has high potential for engineering deployment.关键词
交通工程/匝道控制/DDQN-WRTD算法/深度强化学习/仿真验证Key words
traffic engineering/ramp metering/DDQN-WRTD algorithm/deep reinforcement learning/simulation verification分类
交通工程引用本文复制引用
张玉杰,唐浩铜,徐倩,姚进强,熊辉,徐志刚..面向主线匝道协调的深度强化学习匝道控制方法[J].交通信息与安全,2025,43(6):86-97,12.基金项目
国家自然科学基金重点项目(52432013)、浙江省交通集团技术研发总院有限责任公司科技计划项目(202303)资助 （52432013）
