南京师范大学学报（工程技术版）2023，Vol.23Issue(4)：29-36,8.DOI:10.3969/j.issn.1672-1292.2023.04.004
新型轻量化神经网络结构范式的剪枝研究
Pruning Research on New Lightweight Neural Network Structures Paradigm
宋晨 1魏子重 1姜凯 1李锐 1段强1
作者信息
- 1. 山东浪潮科学研究院有限公司,山东 济南 250014
- 折叠
摘要
Abstract
With the widespread adoption of deep learning technology,the object detection task in image processing has made vigorous progress.Along with the popularity and development of large models,the accuracy of deep learning models continuously improves.However,these large models are difficult to deploy on edge devices that are increasingly developing.To address the current object detection tasks at the edge-side,a network structure combining MobileOne-S0 and SSD is proposed.This network structure is reparameterized to form a VGG-like network structure for the inference process.Then,three different pruning criteria are used,including unstructured weight pruning,structured BN pruning,and Taylor pruning.The results show that weight pruning has the worst effect,while the two structured pruning methods have almost the same decrease rate for FLOPs and parameter quantity with the increase of sparsity.However,the accuracy drop of BN pruning is slower than that of Taylor pruning while Taylor pruning has the best pruning effect on peak memory size.When the model precision decreases by about 10%,BN pruning can compress the parameter quantity by 22.3 times,FLOPs by 9.4 times,and peak memory usage by 2.5 times.The final model size is only 123.88 kB,making it easier to deploy on TinyML-suitable,MCU-level,low-power end-side devices.关键词
MobileOne/SSD/深度可分离卷积/剪枝/TinyMLKey words
MobileOne/SSD/deep separable convolution/pruning/TinyML分类
信息技术与安全科学引用本文复制引用
宋晨,魏子重,姜凯,李锐,段强..新型轻量化神经网络结构范式的剪枝研究[J].南京师范大学学报（工程技术版）,2023,23(4):29-36,8.
