交通信息与安全2024，Vol.42Issue(1)：105-114,10.DOI:10.3963/j.jssn.1674-4861.2024.01.012
基于强化学习的车道级可变限速控制策略
Differential Variable Speed Limit Control Strategy Based on Reinforcement Learning
摘要
Abstract
In addressing the challenges posed by variable traffic conditions within highway merging lanes impacted by merging vehicles,a reinforcement learning(RL)model is developed for differential variable speed limit(DVSL)control.Due to the difficulty of solving the DVSL control problem with high-dimensional action space,this paper optimizes the action space by using the speed limit change value,determines the state space as well as the reward function considering multiple factors;in the solution process,it is improved by using the Prioritized Experience Re-play(PER)technique in order to improve the training efficiency and model performance;and at the same time,it proposes an inter-lane safety detection mechanism to assist the PER-DDQN to unfold the training and ensure the im-plementability of the lane-level variable velocity limit model.Furthermore,the merging area is simulated with SU-MO to examine the performance of the DVSL controller.The results reveal that,compared with the no-control sce-nario,the proposed method yields a 41.88%reduction in overall travel time and a 5.65%increase in average speed.In the merging zone,a notable 66.91%reduction in travel time and a 43.42%increase in average speed are achieved.And the RL based DVSL control strategy effectively minimizes congestion time for each lane due to smoother speed changes.Furthermore,when evaluating the impact of varying penetration scenarios on the proposed method,the RL based DVSL control strategy outperforms the no-control scenario particularly when the penetration of connected-automated vehicles(CAVs)is below 60%.In scenarios with 20%,40%,and 60%penetration rates,the average travel time is reduced by 41.88%,13.38%,and 7.46%,with corresponding average speed improvements of 6.08%,2.36%,and 1.61%,respectively.However,at penetration rate of 80%or higher,there is no significant im-provement in the DVSL control strategy due to the improvement of CAVs to the traffic flow.关键词
智能交通/车道级可变限速/控制策略/强化学习/高速合流区/异质交通流Key words
intelligent traffic/differential variable speed limit control/control strategy/reinforcement learning/high-way merging area/mixed traffic flow分类
交通工程引用本文复制引用
白如玉,焦朋朋,陈越,张瑶..基于强化学习的车道级可变限速控制策略[J].交通信息与安全,2024,42(1):105-114,10.基金项目
国家自然科学基金项目(52172301)、国家社科基金项目(21ZDA029)、北京市社会科学基金项目(21GLA010)资助 （52172301）
