南京邮电大学学报（自然科学版）2024，Vol.44Issue(3)：72-79,8.DOI:10.14132/j.cnki.1673-5439.2024.03.009
基于强关联平滑约束的目标检测模型剪枝方法
Model pruning for object detection via strong correlation smoothing constraints
摘要
Abstract
Although the research on lightweight object detection models has produced many representative results,these models still suffer from a cliff-like decay of detection accuracy when they are pruned at a high ratio.Some researchers find that the fluctuation of the gradient after pruning is the key factor affecting the model performance when exploring the root cause of the pruning performance degradation of the mainstream object detection networks.Therefore,a pruning framework based on gradient self-distillation smoothing is constructed and called as SCSC,a pruning framework with strongly correlation smoothing constraints.First,the historical gradient and the current gradient are defined as the teacher and the student in the self-distillation theory,and the student gradient approaches the teacher gradient as much as possible by imitating the teacher,achieving gradient smoothing.Second,based on the gradient smoothing result,a pruning scheme based on strong correlation constraints is proposed.This scheme forms a strong correlation group with the historical gradient and the current gradient,and enhances the sparsity of the model weight parameters by strengthening the contribution of the historical gradient to the current gradient update.Through the experiments on the PASCAL VOC2007 dataset,SCSC achieves a 2 percentages improvement in average precision compared with mainstream pruning methods;on the KITTI dataset,when the SCSC pruning rate is 80%,the recognition accuracy decay is only decreased 3 percentages from that of the original network.关键词
卷积神经网络/知识蒸馏/模型剪枝Key words
convolutional neural networks/knowledge distillation/model pruning分类
信息技术与安全科学引用本文复制引用
康彬,李卓,邱坤,窦海娥,王磊,郑宝玉..基于强关联平滑约束的目标检测模型剪枝方法[J].南京邮电大学学报（自然科学版）,2024,44(3):72-79,8.基金项目
国家自然科学基金(62171232,62071255,62371253,62001248)、江苏省重点研发计划(BE2023087)和江苏省高校重点项目(20KJA510009)资助项目 （62171232,62071255,62371253,62001248）
