航空兵器2024，Vol.31Issue(6)：50-56,7.DOI:10.12132/ISSN.1673-5048.2024.0130
融合DDQN与示教学习的高超声速飞行器智能姿态控制方法研究
Intelligent Attitude Control of Hypersonic Vehicle Based on DDQN and Deep Q-Learning from Demonstrations
摘要
Abstract
In order to improve the speed and accuracy of solving the attitude control problem of hypersonic vehi-cle,an intelligent attitude control method of hypersonic vehicle based on demonstration learning is proposed.Firstly,the control model of hypersonic vehicle is established,and the appropriate action is selected as the attitude control out-put.Secondly,an algorithm based on DDQN(Double Deep Q-Network)and DQfD(Deep Q-learning from Demonstra-tions)is designed,which divides the training of agents into two stages:pre-training and formal training.In the pre-training stage,the agent extracts small batch data from the demonstration data,and applies four loss functions to update the neural network.In the formal training phase,samples are taken from the data generated by its own training and demonstration data,and the proportion of two types of data in each small batch is controlled through priority experience replay buffer.Learning through interaction with the environment,so that the hypersonic vehicle can adaptively adjust its attitude according to changes in the flight environment.The simulation results show that the reinforcement learning method based on demonstration data can track control command,realize the attitude control of hypersonic vehicle,and improve the performance of neural network in the early stage of training,with a higher average reward.关键词
高超声速飞行器/姿态控制/强化学习/示教学习/DDQNKey words
hypersonic vehicle/attitude control/reinforcement learning/learning from demonstrations/DDQN分类
军事科技引用本文复制引用
刘静文,蔡光斌,凡永华,樊红东,吴彤,尚逸鸣..融合DDQN与示教学习的高超声速飞行器智能姿态控制方法研究[J].航空兵器,2024,31(6):50-56,7.基金项目
国家自然科学基金项目(61773387) （61773387）
