计算机科学与探索2026，Vol.20Issue(9)：2553-2568,16.DOI:10.3778/j.issn.1673-9418.2512003
面向经验质量与效率协同优化的改进MADDPG围捕算法
Improved MADDPG Algorithm for Cooperative Pursuit via Joint Optimization of Experience Quality and Efficiency
摘要
Abstract
To address training oscillations and slow policy convergence caused by poor experience quality and low utiliza-tion efficiency in multi-agent deep reinforcement learning(MARL)for cooperative pursuit tasks,an improved algorithm,STEP-MADDPG(sequence-temporal experience prioritized multi-agent deep deterministic policy gradient)is proposed to synergistically optimize experience quality and efficiency.It constructs a dual-layer optimization framework featuring a"sample denoising and sequence guidance"closed-loop mechanism for the synergistic optimization of experience quality and efficiency.In the experience quality enhancement layer,long short-term memory(LSTM)networks extract temporal dependencies from observation sequences in partially observable environments,while the input structure of value network(Critic)is reconstructed to align the hidden states of policy network(Actor).Furthermore,a ternary decoupled sampling(TDS)strategy mitigates sample distribution conflicts between the Actor and Critic.By integrating the loss-adjusted priori-tized mechanism(LAP)with inverse prioritized sampling,TDS filters high-confidence experiences for the Actor,reducing gradient variance and enhancing training stability.In the experience efficiency optimization layer,prioritized sequence experience replay(PSER)focuses the model on critical experience sequences.Concurrently,an adaptive mixed sampling mechanism dynamically adjusts sample weights for exploration and exploitation during policy updates.The distribution correction effect of this mechanism inversely stabilizes the input distribution of quality layer,reducing temporal feature extraction uncertainty.Simulations conducted in 2D pursuit scenarios involving static and dynamic targets and obstacles demonstrate that the proposed STEP-MADDPG algorithm significantly outperforms baseline algorithms in terms of con-vergence speed and pursuit success rate across all evaluated scenarios.In fully dynamic scenarios,convergence speed im-proves by 29%to 42%,and the pursuit success rate increases by 6%to 25%.关键词
多智能体围捕/深度强化学习/长短期记忆网络/序列优先经验回放/三元解耦采样策略Key words
multi-agent pursuit/deep reinforcement learning/long short-term memory network/sequence-based priori-tized experience replay/ternary decoupled sampling strategy分类
信息技术与安全科学引用本文复制引用
张红强,胡凯,吴亮红,王汐,左词立,陈磊,刘朝华,陈祖国..面向经验质量与效率协同优化的改进MADDPG围捕算法[J].计算机科学与探索,2026,20(9):2553-2568,16.基金项目
国家自然科学基金重点项目(62533010) （62533010）
湖南省教育厅资助科研项目(23A0382) （23A0382）
湖南省教育厅优秀青年项目(23B0459) （23B0459）
海南省自然科学基金(625QN361) （625QN361）
湖南省教育厅科研基金项目(24B0449). This work was supported by the National Natural Science Foundation of China(62533010),the Project Supported by Scientific Re-search Fund of Hunan Provincial Education Department(23A0382),the Excellent Youth Project of Hunan Provincial Education Depart-ment(23B0459),the Natural Science Foundation of Hainan Province(625QN361),and the Scientific Research Fund of Hunan Provin-cial Education Department(24B0449). （24B0449）
