网络与信息安全学报2025，Vol.11Issue(2)：175-188,14.DOI:10.11959/j.issn.2096-109x.2025023
基于多模态特征融合的对抗样本防御方法研究
Research on adversarial examples defense method based on multi-modal feature fusion
摘要
Abstract
In recent years,the vulnerability of neural network models to adversarial attacks has been a significant concern,particularly in image classification tasks,where such attacks can lead to incorrect classifications.To coun-teract these attacks,numerous defence methods have been proposed.Existing defence methods have predominantly concentrated on enhancing model structures or adopting adversarial training methods individually,resulting in a single type of defence and potentially compromising the model's classification capability.Drawing on the human vi-sual system's ability to perceive information through multimodal sensory inputs,a multimodal pyramid feature fu-sion(MPFF)defence method was proposed,integrating textual descriptions of images into the image information.Initially,ViT-GPT2 was utilized to generate corresponding textual descriptions based on image information,while a feature pyramid network captured multi-scale information.Subsequently,a pre-trained TF-IDF model was em-ployed to extract feature matrices from the textual descriptions,and a ResNet50 model was used to extract image features.These image and text features were then weighted and fused to obtain the final multimodal features.Fi-nally,a classifier was applied to perform classification detection using the fused features.Comparative experiments were conducted on the CIFAR-10 and ImageNet datasets.The experimental results demonstrate that the accuracy of the proposed method is improved by 21.8%and 22.5%on average compared to other methods under black-box at-tacks with varying disturbance intensities on the two datasets respectively.关键词
对抗防御/多模态特征融合/特征金字塔网络/特征提取/交叉熵损失Key words
adversarial defense/multimodal feature fusion/feature pyramid network/feature extraction/cross en-tropy loss分类
信息技术与安全科学引用本文复制引用
魏宣宣,刘万平,卢玲..基于多模态特征融合的对抗样本防御方法研究[J].网络与信息安全学报,2025,11(2):175-188,14.基金项目
重庆市自然科学基金(cstc2021jcyj-msxmX0594) （cstc2021jcyj-msxmX0594）
重庆理工大学研究生教育高质量发展行动计划(gzlcx20243171) The Chongqing Natural Science Foundation(cstc2021jcyj-msxmX0594),Action Plan for High-Quality Development of Graduate Education of Chongqing University of Technology(gzlcx20243171) （gzlcx20243171）
