一种基于后门技术的深度强化学习水印框架OACSTPCD

Deep Reinforcement Learning(DRL)has demonstrated its effectiveness in various complex tasks.Its outstanding performance is rapidly accelerating its commercialization.Generating a DRL model requires substantial computational resources and expertise,making a well-trained DRL model the core intellectual property of artificial intelligence applications and products.Protecting these DRL models from illegal plagiarism,unauthorized distribution,and copying is crucial.We propose a DRL watermarking framework,DrlWF,based on backdoor technology to protect the intellectual property of DRL models.It uses a new evaluation metric,watermark action execution rate,to measure watermarking performance.We embed a watermark pattern into the training state and use watermark states with this watermark pattern to train the model.The watermark embedding operation in our scheme can be achieved through minimal data watermarking(only 0.025%of train-ing data)and reward modification without affecting model performance.Experimental results demon-strate that the DRL model performs well under normal conditions.However,its performance drops sharply to less than 1%of its original performance under watermarked conditions,with a watermark action execution rate of 99%.The ownership of the model can be verified by the sharp decline in performance and the model's response towards watermark states.In addition,the DRL watermark proposed in this paper has good robustness,as the model can still recognize the watermark and maintain a watermark action execution rate of over 99%,even under model fine-tuning and com-pression.