电子学报2025，Vol.53Issue(2)：287-300,14.DOI:10.12263/DZXB.20240518
云边协同大模型块粒度重训方法
Cloud-Edge Collaborative Retraining of Foundation Models at the Block Granularity
摘要
Abstract
Foundation models deployed in dynamic edge environment encounter continuously evolving input data dis-tributions,requiring retraining them to maintain high accuracy.However,existing retraining techniques can only train fixed compressed models within the constraints of device resources and retraining windows,thus considerably lowering accura-cies due to these small models'limited generalization ability.For such an issue,this paper proposes BlockTrainer,an edge-cloud collaborative retraining approach of foundation models at the block granularity.BlockTrainer first introduces a model retraining scaling law to evaluate the accuracy contributions of different blocks in a foundation model according to its latest input data at edge.Based on this evaluation,it generates the optimal retraining solution under resource constraints,and dy-namically converts the most accuracy-relevant parts of the model into retrainable small models at edge,thereby constructing a collaborative training system between large and small models.Comparative experiments on real edge-cloud platforms show that BlockTrainer improves the retraining accuracy of foundation models by 81.24%using the same resource con-sumptions,and supports retraining a model of up to 33 billion parameters.关键词
大模型/边缘侧动态环境/模型重训/缩放定律/云边大小模型协同训练Key words
foundation model/dynamic environment at edge/model retraining/scaling law/edge-cloud collabora-tive retraining of large and small models分类
计算机与自动化引用本文复制引用
张青龙,韩锐,刘驰..云边协同大模型块粒度重训方法[J].电子学报,2025,53(2):287-300,14.基金项目
国家重点研发计划(No.2023YFE0209100) （No.2023YFE0209100）
国家自然科学基金(No.62272046,No.62132019,No.61872337) National Key Research and Development Program of China(No.2023YFE0209100) （No.62272046,No.62132019,No.61872337）
National Natural Science Foundation of China(No.62272046,No.62132019,No.61872337) （No.62272046,No.62132019,No.61872337）
