电子科技2025，Vol.38Issue(10)：1-9,9.DOI:10.16180/j.cnki.issn1007-7820.2025.10.001
一种随机掩膜和自适应特征蒸馏算法
A Random Mask and Adaptive Feature Distillation Algorithm
摘要
Abstract
In view of the problems that the feature connection between teacher and student relies on manual de-sign in existing feature distillation methods,and it is difficult to determine the distillation strength between features,which leads to the students'model learning useless information,a MAFD(Random Mask and Adaptive Feature Distil-lation)is proposed in this study.This algorithm adaptively determines the distillation strength between teacher-student candidate feature layers by introducing a self-attention mechanism.In the stage of student feature generation,the random pixel mask strategy is introduced to make the teacher model guide student feature generation,so as to im-prove the representativeness of the remaining pixels and enhance the representation ability of the student network.The experimental results show that the knowledge distillation network improves the performance of the student model rela-tive to the baseline by 2.0～6.2 percentage points on the CIFAR100 and ImageNet data sets.The improvement of CUB-200,indoor,Actions and Dogs is 27.27 percentage points,14.75 percentage points,25.55 percentage points and 12.55 percentage points,respectively,when compared with the baseline.The improved performance of the Reti-naNet model on the COCO-2017 data set is verified,showing that MAFD can better reduce the loss of knowledge transfer between the teacher model and the student model.关键词
特征蒸馏/模型轻量化/卷积神经网络/计算机视觉/深度学习/掩膜策略/自适应特征连接Key words
feature distillation/model light-weighting/convolutional neural networks/computer vision/deep learning/masking strategy/adaptive feature connectivity分类
信息技术与安全科学引用本文复制引用
冯健,吴鹏..一种随机掩膜和自适应特征蒸馏算法[J].电子科技,2025,38(10):1-9,9.基金项目
浙江省自然科学基金(LY21F010016)Natural Science Foundation of Zhejiang(LY21F010016) （LY21F010016）
