自动化学报2017，Vol.43Issue(7)：1142-1159,18.DOI:10.16383/j.aas.2017.c160325
基于权值动量的RBM加速学习算法研究
Research on RBM Accelerating Learning Algorithm with Weight Momentum
摘要
Abstract
Momentum algorithms can accelerate the training speed of restricted Boltzmann machine theoretically.Through a simulation study on existing momentum algorithms,it is found that existing momentum algorithms for training restricted Boltzmann machine have a poor accelerating effect and they began to lose acceleration performance.In the latter part of training process.Focusing on this problem,firstly,this paper gives a theoretical analysis of the algorithms based on Gibbs sampling convergence theorem.It is proved that the acceleration effect of existing momentum algorithms is at the expense of enlarging network weights.Then,a further investigation on network weights shows that the network weights contain a lot of information of the true gradient direction which can be used to train the network.According to this,a weight momentum algorithm is proposed based on the weight of the network.Finally,simulation results demon strate that the proposed algorithm has a better acceleration effect and has the accelerating ability even in the end of the training process.Therefore the proposed algorithm can well make up for the weaknesses of existing momentum algorithms.关键词
深度学习/受限玻尔兹曼机/动量算法/权值动量Key words
Deep learning/restricted Boltzmann machine (RBM)/momentum algorithm/weight momentum引用本文复制引用
李飞,高晓光,万开方..基于权值动量的RBM加速学习算法研究[J].自动化学报,2017,43(7):1142-1159,18.基金项目
国家自然科学基金(61305133,61573285)资助 Supported by National Natural Science Foundation of China(61305133,61573285) （61305133,61573285）
