北京交通大学学报2024，Vol.48Issue(5)：1-9,9.DOI:10.11860/j.issn.1673-0291.20230074
基于多智能体深度强化学习的多无人机辅助移动边缘计算轨迹设计
Trajectory design for multi-UAV-assisted mobile edge computing based on multi-agent deep reinforcement learning
摘要
Abstract
Unmanned Aerial Vehicle(UAV)-assisted Mobile Edge Computing(MEC)networks can provide high-quality computational services to ground User Equipment(UE),but real-time trajectory design for multiple UAVs remains a significant challenge.To address this issue,a trajectory design al-gorithm based on multi-agent deep reinforcement learning is proposed,utilizing the Multi-Agent Deep Deterministic Policy Gradient(MADDPG)framework to collaboratively design UAV trajectories.Considering the limited battery capacity of UAVs,a critical constraint on UAV network performance,the optimization problem is formulated to improve the sum of UAV energy efficiencies.This involves jointly optimizing the trajectories of UAV clusters and the offloading decisions of UEs.Each agent in-teracts with the edge computing network environment,observes its local state,and determines trajec-tory coordinates via an Actor network.The Critic network is trained by incorporating the action and ob-servation of other agents,thereby refining the trajectory policy generated by the Actor network.Simu-lation results demonstrate that the MADDPG-based trajectory design algorithm exhibits excellent con-vergence and robustness,significantly enhancing UAV energy efficiency.Specifically,the proposed al-gorithm outperforms the random flight algorithm by 120%at most,the circular flight algorithm by 20%at most,and the Deep Deterministic Policy Gradient(DDPG)algorithm by 5%to 10%.关键词
无人机轨迹设计/移动边缘计算/强化学习/多智能体深度确定性策略梯度Key words
UAV trajectory design/Mobile Edge Computing(MEC)/reinforcement learning/Multi-Agent Deep Deterministic Policy Gradient(MADDPG)分类
信息技术与安全科学引用本文复制引用
徐少毅,杨磊..基于多智能体深度强化学习的多无人机辅助移动边缘计算轨迹设计[J].北京交通大学学报,2024,48(5):1-9,9.基金项目
国家重点研发计划(2022YFB3303702) （2022YFB3303702）
国家自然科学基金(61931001) National Key R&D Plan(2022YFB3303702) （61931001）
National Natural Science Foundation of China(61931001) （61931001）
