计算机科学与探索2025，Vol.19Issue(4)：835-853,19.DOI:10.3778/j.issn.1673-9418.2403071
神经网络轻量化综述
Review of Neural Network Lightweight
摘要
Abstract
With the continuous progress of deep learning technology,artificial neural network models have shown unprec-edented performance in many fields such as image recognition,natural language processing,and autonomous driving.These models often have millions or even billions of parameters and learn complex feature representations through large amounts of training data.However,in resource-constrained environments,such as mobile devices,embedded systems and other edge computing scenarios,the power consumption,memory usage and computing efficiency of the model limit the application of large-scale neural network models.To solve this problem,the researchers have proposed a variety of model compression techniques,such as pruning,distillation,neural network search(NAS),quantization,and low-rank decompo-sition,which aim to reduce the number of parameters,computational complexity,and storage requirements of the model,while maintaining the accuracy of the model as much as possible.The following is a systematic introduction to the devel-opment process of these model compression methods,focusing on the main principles and key technologies of each method.It mainly includes different strategies of pruning techniques,such as structured pruning and unstructured pruning;how to define knowledge in knowledge distillation;search space,search algorithm and network performance evaluation in NAS;post-training quantization and in-training quantization in quantization;and the singular value decomposition and tensor decomposition in low rank decomposition.Finally,the future development direction of model compression technology is discussed.关键词
剪枝/量化/知识蒸馏/神经网络搜索(NAS)/低秩分解Key words
pruning/quantization/knowledge distillation/neural network search(NAS)/low-rank decomposition分类
计算机与自动化引用本文复制引用
段宇晨,方振宇,郑江滨..神经网络轻量化综述[J].计算机科学与探索,2025,19(4):835-853,19.基金项目
国家自然科学基金(62202385f) （62202385f）
中央高校基本科研业务费专项资金(G2021KY05103).This work was supported by the National Natural Science Foundation of China(62202385f),and the Basic Research Funds for Central Universities of China(G2021KY05103). （G2021KY05103）
