计算机应用研究2023，Vol.40Issue(12)：3655-3659,3665,6.DOI:10.19734/j.issn.1001-3695.2023.04.0177
带有随机改进Barzilai-Borwein步长的小批量稀疏随机方差缩减梯度法
Mini-batch sparse stochastic variance reduced gradient method with randomly improved Barzilai-Borwein steps
摘要
Abstract
In order to better cope with large-scale high-dimensional sparse data sets in today's era,this paper combined the advantages of BB method,mini-batch algorithm and stochastic variance reduced gradient(SVRG),and proposed a mini-batch sparse stochastic variance reduced gradient method(MSSVRG-R2BB)with randomly improved Barzilai-Borwein step size.Firstly,on the basis of calculating the total gradient in the outer loop of SVRG,this paper added the subgradient of L,norm,designed a sparse approximate gradient for the inner loop of SVRG,and obtained a sparse SVRG algorithm(SSVRG).On this basis,it proposed to use the improved BB method of random selection to automatically calculate and update the step size in the mini-batch sparse stochastic variance reduced gradient method,which solved the step size selection problem of the mini-batch algorithm,and expanded the MSSVRG-R2BB algorithm.Numerical experiments show that MSSVRG-R2BB algorithm can not only reduce the operation cost and reach the convergence upper bound faster,but also achieve the same optimization level of other advanced small-batch algorithms,and perform stably and well for different initial parameter selection.关键词
随机梯度下降法/小批量算法/Barzilai-Borwein方法/方差缩减/凸优化Key words
stochastic gradient descent/small batch algorithm/Barzilai-Borwein method/variance reduction/convex optimi-zation分类
信息技术与安全科学引用本文复制引用
秦传东,杨旭..带有随机改进Barzilai-Borwein步长的小批量稀疏随机方差缩减梯度法[J].计算机应用研究,2023,40(12):3655-3659,3665,6.基金项目
宁夏自然科学基金一般项目(2021AAC03230) （2021AAC03230）
