现代电子技术2025，Vol.48Issue(13)：77-82,6.DOI:10.16652/j.issn.1004-373x.2025.13.012
基于样本重要性的分布式深度学习通信优化策略
Distributed deep learning communication optimization strategy based on sample importance
蒙玉功1
作者信息
- 1. 广西码农信息科技有限公司,广西 南宁 530003
- 折叠
摘要
Abstract
The computing nodes in distributed deep learning(DDL)need to frequently exchange gradient data with the server,which results in large communication overhead.In view of this,a DDL communication optimization strategy based on sample importance is proposed.It mainly includes three contents.The importance distribution of data samples is explored by confirmatory experiments.The importance of data samples is evaluated by cross-entropy loss.In combination with the network status awareness mechanism and by taking the end-to-end network delay as the network status feedback indicator,the computing nodes are used to adjust the compression ratios of the transmission gradient dynamically,which reduces network traffic while ensuring model convergence,thereby improving the training efficiency of DDL.Experimental results show that the proposed method can improve communication efficiency effectively in distributed training scenarios of different scales.In comparison with the existing gradient compression strategies,the proposed method can reduce distributed training time by up to 40%.关键词
分布式深度学习/随机梯度下降/样本重要性/交叉熵/网络状态感知/动态压缩Key words
DDL/stochastic gradient descent/sample importance/cross-entropy/network status awareness/dynamic compression分类
信息技术与安全科学引用本文复制引用
蒙玉功..基于样本重要性的分布式深度学习通信优化策略[J].现代电子技术,2025,48(13):77-82,6.
