计算机工程2025，Vol.51Issue(10)：1-17,17.DOI:10.19678/j.issn.1000-3428.0070575
基于预训练的持续学习方法综述(特邀)
Survey of Pre-training-based Continual Learning Methods(Invited)
摘要
Abstract
Traditional machine learning algorithms perform well only when the training and testing sets are identically distributed.They cannot perform incremental learning for new categories or tasks that were not present in the original training set.Continual learning enables models to learn new knowledge adaptively while preventing the forgetting of old tasks.However,they still face challenges related to computation,storage overhead,and performance stability.Recent advances in pre-training models have provided new research directions for continual learning,which are promising for further performance improvements.This survey summarizes existing pre-training-based continual learning methods.According to the anti-forgetting mechanism,they are categorized into five types:methods based on prompt pools,methods with slow parameter updating,methods based on backbone branch extension,methods based on parameter regularization,and methods based on classifier design.Additionally,these methods are classified according to the number of phases,fine-tuning approaches,and use of language modalities.Subsequently,the overall challenges of continual learning methods are analyzed,and the applicable scenarios and limitations of various continual learning methods are summarized.The main characteristics and advantages of each method are also outlined.Comprehensive experiments are conducted on multiple benchmarks,followed by in-depth discussions on the performance gaps among the different methods.Finally,the survey discusses research trends in pre-training-based continual learning methods.关键词
持续学习/灾难性遗忘/预训练模型/高效参数微调/深度神经网络Key words
continual learning/catastrophic forgetting/pre-training model/efficient parameter fine-tuning/deep neural network分类
计算机与自动化引用本文复制引用
路悦,周翔宇,张世周,梁国强,邢颖慧,程德,张艳宁..基于预训练的持续学习方法综述(特邀)[J].计算机工程,2025,51(10):1-17,17.基金项目
国家自然科学基金(62201467) （62201467）
中国博士后科学基金(2022TQ0260,2023M742842) （2022TQ0260,2023M742842）
西安市科协青年人才托举计划(959202313088) （959202313088）
陕西省创新能力支撑计划(2024ZC-KJXX-043) （2024ZC-KJXX-043）
陕西省自然科学基础研究计划(2022JC-DW-08). （2022JC-DW-08）
