福建师范大学学报（自然科学版）2024，Vol.40Issue(1)：96-105,10.DOI:10.12046/j.issn.1000-5277.2024.01.011
一种基于后门技术的深度强化学习水印框架
A Deep Reinforcement Learning Watermarking Framework Based on Backdoor Technology
摘要
Abstract
Deep Reinforcement Learning(DRL)has demonstrated its effectiveness in various complex tasks.Its outstanding performance is rapidly accelerating its commercialization.Generating a DRL model requires substantial computational resources and expertise,making a well-trained DRL model the core intellectual property of artificial intelligence applications and products.Protecting these DRL models from illegal plagiarism,unauthorized distribution,and copying is crucial.We propose a DRL watermarking framework,DrlWF,based on backdoor technology to protect the intellectual property of DRL models.It uses a new evaluation metric,watermark action execution rate,to measure watermarking performance.We embed a watermark pattern into the training state and use watermark states with this watermark pattern to train the model.The watermark embedding operation in our scheme can be achieved through minimal data watermarking(only 0.025%of train-ing data)and reward modification without affecting model performance.Experimental results demon-strate that the DRL model performs well under normal conditions.However,its performance drops sharply to less than 1%of its original performance under watermarked conditions,with a watermark action execution rate of 99%.The ownership of the model can be verified by the sharp decline in performance and the model's response towards watermark states.In addition,the DRL watermark proposed in this paper has good robustness,as the model can still recognize the watermark and maintain a watermark action execution rate of over 99%,even under model fine-tuning and com-pression.关键词
深度强化学习/知识产权保护/后门攻击/神经网络水印/黑盒模型Key words
deep reinforcement learning/intellectual property protection/backdoor/neural network watermarking/black-box model分类
信息技术与安全科学引用本文复制引用
陈瑜霖,姚志强,金彪,李璇,蔡娟娟,熊金波..一种基于后门技术的深度强化学习水印框架[J].福建师范大学学报（自然科学版）,2024,40(1):96-105,10.基金项目
国家自然科学基金资助项目(62272103、62272102) （62272103、62272102）
福建省自然科学基金资助项目(2023J01531) （2023J01531）
福建省教育厅中青年教师科研项目(JAT220045) （JAT220045）
