南京理工大学学报（自然科学版）2025，Vol.49Issue(2)：155-166,12.DOI:10.14177/j.cnki.32-1397n.2025.49.02.003
基于数据压缩和梯度追踪的方差缩减的联邦优化算法
A variance reduced federated optimization algorithm with data compression and gradient tracking
摘要
Abstract
To overcome the challenges of computational cost,communication cost,and data heterogeneity in federated learning,we proposed a variance reduced variant of federated optimization with compression and gradient tracking(FedCOMGATE-VR).Unlike traditional federated learning algorithms that rely on simple stochastic gradient estimations,FedCOMGATE-VR incorporates variance reduced stochastic gradient estimators,enabling the use of larger step sizes to accelerate convergence.Additionally,it employs data compression technique to process uploaded model parameters,significantly reducing communication cost.Furthermore,by tracking the deviation between local and global gradients using gradient tracking techniques,it effectively addresses the challenges posed by data heterogeneity in federated learning scenarios.Theoretically,the sublinear convergence rate for the algorithm under non-convex setting and linear convergence rate under strongly convex setting are given.Furthermore,FedCOMGATE-VR was applied to the classification task on the Fashion-MNIST and CIFAR-10 datasets,with comparative experiments conducted against existing algorithms under various parameter settings(such as step size and local update frequency).Experimental results demonstrate that FedCOMGATE-VR is well-suited to complex heterogeneous data environments.Specifically,when achieving the same target training accuracy,the algorithm reduces communication rounds by approximately 20%and total iterations by around 66%compared to FedCOMGATE,effectively lowering both communication and computational costs.关键词
联邦学习/随机梯度下降/方差缩减/数据异质Key words
federated learning/stochastic gradient descent/variance reduction/data heterogeneity分类
计算机与自动化引用本文复制引用
贾泽慧,李登辉,刘治宇,黄洁茹..基于数据压缩和梯度追踪的方差缩减的联邦优化算法[J].南京理工大学学报（自然科学版）,2025,49(2):155-166,12.基金项目
国家自然科学基金(11801279) （11801279）
江苏省自然科学基金(BK20180782) （BK20180782）
