郑州大学学报（工学版）2025，Vol.46Issue(5)：60-68,9.DOI:10.13705/j.issn.1671-6833.2024.02.011
基于深度强化学习NoisyNet-A3C算法的自动化渗透测试方法
Automated Penetration Testing Method Based on Deep Reinforcement Learning NoisyNet-A3C Algorithm
摘要
Abstract
In the field of automated penetration testing,most existing attack path decision algorithms are based on partially observable Markov decision processes(POMDP),with problems such as high algorithm complexity,slow convergence speed,and susceptibility to getting stuck in local optima.In this study a reinforcement learning algo-rithm NoisyNet-A3C was proposed based on Markov Decision Process(MDP).And it was applied to the field of automated penetration testing.This algorithm trained actor-critic through multiple threads,and the operation results of each thread were fed back to the main neural network.At the same time,the latest parameter updates were ob-tained from the main neural network,fully utilizing computer performance,reducing data correlation,and impro-ving training efficiency.In addition,adding noise parameters and weight network training update parameters to the training network increased the randomness of the behavior strategy,facilitated faster exploration of effective paths,reduced the impact of data disturbances,and enhanced the robustness of the algorithm.The experimental results showed that compared with A3C,Q-learning,DQN,and NDSPI-DQN algorithms,the NoisyNet-A3C algorithm converged more than 30%faster,verifying that the algorithm proposed in this study converged faster.关键词
渗透测试/攻击路径决策/A3C算法/深度强化学习/MetasploitKey words
penetration testing/attack path decision/A3C algorithm/deep reinforcement learning/Metasploit分类
信息技术与安全科学引用本文复制引用
董卫宇,刘鹏坤,刘春玲,唐永鹤,马钰普..基于深度强化学习NoisyNet-A3C算法的自动化渗透测试方法[J].郑州大学学报（工学版）,2025,46(5):60-68,9.基金项目
国家重点研发计划项目(2018YFB080∗∗∗) （2018YFB080∗∗∗）
河南省重点研发项目(221111210300) （221111210300）
