计算机与现代化Issue(4)：42-49,8.DOI:10.3969/j.issn.1006-2475.2025.04.007
基于图神经网络的多智能体强化学习对抗策略检测算法
Graph Neural Network-based Multi-agent Reinforcement Learning for Adversarial Policy Detection Algorithm
摘要
Abstract
In a multi-agent environment,the reinforcement learning model has security vulnerabilities in coping with adversarial attacks and is susceptible to adversarial attacks,of which adversarial policy-based adversarial attacks are more difficult to de-fend against because they do not directly modify the victim's observations.To solve this problem,this paper proposes a graph neural network-based adversarial policy detection algorithm,which aims to effectively identify malicious behaviors among agents.This paper detects adversarial policy by training the graph neural network as an adversarial policy detector by employing alternative adversarial policies during the collaboration process of the agents,and calculates the trust scores of the other agent based on the local observations of the agents.The detection method in this paper provides two levels of granularity;adversarial detection at the game level detects adversarial policies with very high accuracy,and time-step level adversarial detection allows for adversarial detection at the early stage of the game and timely detection of adversarial attacks.This paper conducts a series of experiments on the StarCraft platform,whose results show that the detection method proposed in this paper can achieve an AUC value of up to 1.0 in detecting the most advanced adversarial policy-based adversarial attacks,which is better than the state-of-the-art detection methods.The detection method in this paper can detect adversarial policy faster than existing methods,and can detect the adversarial attack at the 5th time step at the earliest.Applying this paper's detection method to adversarial defense im-proves the win rate of the attacked game by up to 61 percentage points.In addition experimental results show that the algorithm in this paper is highly generalizable and the detection method in this paper does not need to be trained again and can be directly used to detect observation-based adversarial attacks.Therefore,the method proposed in this paper provides an effective adver-sarial attack detection mechanism for reinforcement learning models in a multi-agent environment.关键词
强化学习/多智能体系统/对抗攻击/对抗检测/图神经网络Key words
reinforcement learning/multi-agent system/adversarial attack/adversarial detection/graph neural network分类
信息技术与安全科学引用本文复制引用
孙启宁,桂智明,刘艳芳,范鑫鑫,路云峰..基于图神经网络的多智能体强化学习对抗策略检测算法[J].计算机与现代化,2025,(4):42-49,8.基金项目
复杂关键软件全国重点实验室自主课题(SKLSDE-2023ZX-17) （SKLSDE-2023ZX-17）
