火力与指挥控制2024，Vol.49Issue(6)：193-199,207,8.DOI:10.3969/j.issn.1002-0640.2024.06.026
一种基于多层特征对齐的知识蒸馏方法
A Knowledge Distillation Method Based on Multi-layer Feature Alignment
闫泽阳 1张宏伟 1王子珍 1彭晴晴 1魏文豪1
作者信息
- 1. 北方自动控制技术研究所,太原 030006
- 折叠
摘要
Abstract
Real-time target detection algorithms(e.g.,YOLO)are designed to efficiently perform object detection tasks on resource-limited edge devices.However,their detection performance is often limited.To address this challenge,a knowledge distillation method based on multi-layer feature alignment is proposed.In order to effectively retain the knowledge in the original data,a distillation metric that incorporates multiple intermediate layers of knowledge from the teacher and student models is introduced,and an alignment weighting factor is incorporated based on the differences between the intermediate layer features of the teacher model and the student model during the training process.Compared with existing knowledge distillation methods,this method enables the student model to learn more useful knowledge from the middle layer of the teacher model.The refined knowledge is used to incrementally train the existing model,avoiding the resource overhead of training multiple independent models.Through experimental comparisons under different scenarios and conditions,this method effectively improves the accuracy of target recognition while reducing the computational and storage costs of the models.The experimental analyses show that the proposed multilayer feature alignment distillation algorithm based on the YOLO model is validated by the COCO2017 dataset,the detection accuracy of the student model is improved from 33.3 to 40.7,the detection accuracy of the model is effectively improved.关键词
知识蒸馏/YOLO算法/多层特征对齐/目标检测Key words
knowledge distillation/YOLO algorithm/multi-layer feature alignment/object detection分类
信息技术与安全科学引用本文复制引用
闫泽阳,张宏伟,王子珍,彭晴晴,魏文豪..一种基于多层特征对齐的知识蒸馏方法[J].火力与指挥控制,2024,49(6):193-199,207,8.
