交通信息与安全2025，Vol.43Issue(2)：95-108,14.DOI:10.3963/j.jssn.1674-4861.2025.02.011
混行下CAV作业区分段式深度强化学习合流模型
A Merging Model Based on Piecewise Deep Reinforcement Learning for Connected and Autonomous Vehicle in Work Zone under Mixed Autonomy
摘要
Abstract
The classical early and late merge work worse under dynamic demand,and render conflict merging gap due to large speed differences at the upstream.To this end,a piece-wise deep reinforcement learning-based merging model is proposed for connected and autonomous vehicles(CAVs)in work zones under mixed autonomy.Above all,the merging conflicts and efficiency reduction caused by many vehicles in closed lanes trying to merge into one gap on the open lane are addressed by the model with speed guidance,gap creation,and positional alignment.Such a model consists of the soft Actor-Critic algorithm-based longitudinal control and the rule-based lane-changing deci-sion-making.For longitudinal control,9 features are selected as the agent state to describe surrounding traffic condi-tions from both local and global views.The mentioned features include the speed and acceleration of the ego vehi-cle,the speed of and the distance to the lead vehicle,the speed of and the distance to the lead and lag vehicles on the adjacent left lane,and the distance to the merging point.Subsequently,a piecewise reward function for CAVs in the work zone is established by optimizing comfort,safety,and efficiency simultaneously.Such a reward function com-bines minimizing acceleration and jerk,preventing collisions,generating merging gaps,aligning with the gap center on the open lane,mitigating vehicular speed differences,adhering to advisory speed,and encouraging following ve-hicles with yield behavior.Particularly,an item of reward function with respect to driving efficiency is shaped on the basis of the speed difference between the lag vehicle on the adjacent lane and the ego vehicle,such that halting of both the CAV and the human-driving vehicle can be alleviated at the merging point.Simulation results illustrate that the proposed model increases by about 4.76%of average speed,and 19.71%of minimal time-to-collision under medium/heavy demand in work zone,in contrast to early merge,late merge and New England merge.In addition,the average speed,minimum time-to-collision,and successful merging rate in mixed autonomy with heterogeneous human-driving vehicles,increase with the increase of the CAV market penetration rate,while all the vehicles merge without halting.关键词
智能交通/作业区合流/合流控制模型/柔性演员-评论家算法/混合交通流Key words
intelligent transportation/work zone merging/merging control model/soft Actor-Critic algorithm/mixed traffic flow分类
交通工程引用本文复制引用
辛琪,荚胜琪,徐猛,齐嘉乐,袁伟..混行下CAV作业区分段式深度强化学习合流模型[J].交通信息与安全,2025,43(2):95-108,14.基金项目
国家自然科学基金项目(52002035)、陕西省重点研发计划(2024CY2-GJHX-87)项目、陕西省自然科学基础研究计划项目(2025JC-YBMS-395)资助 （52002035）
