电子科技大学学报2024，Vol.53Issue(1)：50-59,10.DOI:10.12178/1001-0548.2022285
深度强化学习下连续和离散相位RIS毫米波通信
Continuous vs Discrete:Phase Performance Comparison of RIS-Assisted Millimeter Wave Communication Based on Deep Reinforcement Learning
摘要
Abstract
In this paper,in the distributed Reconfigurable Intelligence Surface(RIS)assisted multi-user millimeter wave(mmWave)system,the deep reinforcement learning(DRL)theory is used to learn and adjust transmit beamforming matrix at the base station and phase shift matrix at the RIS,and jointly optimize the transmit beamforming matrix and phase shift matrix to maximize the weighted sum-rate.Specifically,in the discrete action space,we first design the power codebook and the phase codebook,and propose the Deep Q Network(DQN)algorithm to optimize the beamforming matrix and phase shift matrix;then,in the continuous action space,the Twin Delayed Deep Deterministic(TD3)policy gradient algorithm is used to optimize the beamforming matrix and phase shift matrix.The weighted sum-rates of the system in discrete action space and continuous action space with different number of codebook bits are compare through simulation.In addition,compared with the traditional convex optimization algorithm and the zero-forcing precoding with a random PBF algorithm,the sum-rate performance of DRL algorithm is significantly improved,and the sum-rate of the continuous TD3 algorithm exceeds the convex optimization algorithm by 23.89%,and the performance of the discrete DQN algorithm exceeds the traditional convex optimization algorithm when the number of codebook bits is 4.关键词
深度Q网络(DQN)/深度强化学习/双延迟策略梯度/毫米波/智能反射面Key words
deep Q network(DQN)/deep reinforcement learning/delayed deep deterministic policy gradient/millimeter wave/reconfigurable intelligence surface分类
信息技术与安全科学引用本文复制引用
胡浪涛,杨瑞,刘全金,吴建岚,嵇文,吴磊..深度强化学习下连续和离散相位RIS毫米波通信[J].电子科技大学学报,2024,53(1):50-59,10.基金项目
国家自然科学基金(62171002) （62171002）
安徽省教育厅自然科学基金(KJ2020A0497) （KJ2020A0497）
