自动化学报2023，Vol.49Issue(12)：2481-2492,12.DOI:10.16383/j.aas.c210555
基于滚动时域强化学习的智能车辆侧向控制算法
Receding Horizon Reinforcement Learning Algorithm for Lateral Control of Intelligent Vehicles
摘要
Abstract
This paper presents a receding horizon reinforcement learning(RHRL)algorithm for realizing high-accur-acy lateral control of intelligent vehicles.The overall lateral control is composed of a feedforward control term that is directly computed using the curvature of the reference path and the dynamic model,and a feedback control term that is generated by solving an optimal control problem using the proposed RHRL algorithm.The proposed RHRL adopts a receding horizon optimization mechanism,and decomposes the infinite-horizon optimal control problem in-to several finite-horizon ones to be solved.Different from existing finite-horizon actor-critic learning algorithms,in each prediction horizon of RHRL,a time-independent actor-critic structure is utilized to learn the optimal value function and control policy.Also,compared with model predictive control(MPC),the control learned by RHRL is an explicit state-feedback control policy,which can be deployed directly offline or learned and deployed synchron-ously online.Moreover,the convergence of the proposed RHRL algorithm in each prediction horizon is proven and the stability analysis of the closed-loop system is peroformed.Simulation studies on a structural road show that,the proposed RHRL algorithm performs better than current state-of-the-art methods.The experimental studies on an intelligent driving platform built with a Hongqi E-HS3 electric car show that RHRL performs better than the pure pursuit method in the adopted structural city road scenario,and exhibits strong adaptability to road conditions and satisfactory control performance in the country road scenario.关键词
滚动时域/强化学习/智能汽车/侧向控制Key words
Receding horizon/reinforcement learning/intelligent vehicles/lateral control引用本文复制引用
张兴龙,陆阳,李文璋,徐昕..基于滚动时域强化学习的智能车辆侧向控制算法[J].自动化学报,2023,49(12):2481-2492,12.基金项目
国家重点研究发展计划(2018YFB1305105),国家自然科学基金(62003361,61825305)资助Supported by National Key Research and Development Pro-gram of China(2018YFB1305105)and National Natural Science Foundation of China(62003361,61825305) （2018YFB1305105）
