电讯技术2025，Vol.65Issue(2)：283-292,10.DOI:10.20079/j.issn.1001-893x.231117001
基于多智能体深度强化学习的水声网络资源分配
Multi-agent Deep Reinforcement Learning Based Resources Allocation for Underwater Acoustic Networks
摘要
Abstract
In resource limited underwater acoustic networks,the network capacity and energy efficiency can be improved by using soft frequency reuse technology and adaptive resource allocation technology.However,the underwater acoustic channel has long propagation delays and time-varying features,resulting in the feedback channel state information(CSI)used in adaptive techniques being time-varying and outdated.Imperfect feedback CSI will reduce the performance of adaptive systems.To address this issue,a multi-agent deep Q network based resource allocation(MADQN-RA)method is proposed.The method treats the underwater acoustic soft frequency reuse network as a multi-agent system and employs outdated feedback CSI sequences as the system states.By establishing an effective reward expression,agents can track the properties of time-varying delay underwater acoustic channels and make corresponding resource allocation decisions.To further improve the decision-making accuracy of agents and avoid the partial learning cost of increasing state space dimensions,the MADQN-RA is improved by dynamic state length method.The simulation results show that the system performance achieved through the proposed methods surpasses that of other learning based and channel prediction based methods and converges closer to the theoretically optimal values.关键词
水声网络/资源分配/反馈信道状态信息/多智能体深度Q网络/动态状态长度Key words
underwater acoustic networks/resource allocation/feedback channel state information/multi-agent deep Q network/dynamic state length分类
信息技术与安全科学引用本文复制引用
李梦凡,张育芝,韩翔,冯晓美..基于多智能体深度强化学习的水声网络资源分配[J].电讯技术,2025,65(2):283-292,10.基金项目
国家自然科学基金资助项目(61801372) （61801372）
陕西省教育厅科研计划项目(22JK0454) （22JK0454）
