自动化学报2017，Vol.43Issue(5)：753-764,12.DOI:10.16383/j.aas.2017.c160326
基于改进并行回火算法的RBM网络训练研究
Research on RBM Networks Training Based on Improved Parallel Tempering Algorithm
摘要
Abstract
Currently, most algorithms for training restricted Boltzmann machines (RBMs) are based on multi-step Gibbs sampling. When the sampling algorithm is used to calculate gradient, the sampling gradient is an approximate value of the true gradient, and there is a big error between the sampling gradient and the true gradient, which seriously affects training effect of network. This article focuses on the problems mentioned above. Firstly, numerical error and direction error between gradient and true gradient sampling are analyzed, as well as their influences on the performance of network training. The problems are theoretically analyzed from the angle of Markov sampling. Then a gradient modification model is established to adjust the numerical value and direction of sampling gradient. Furthermore, improved tempering learning based algorithm is put forward, that is, GFPT (Gradient fixing parallel tempering) algorithm. Finally, a comparative experiment on the GFPT algorithm and existing algorithms is given. It demonstrated that GFPT algorithm can greatly reduce the sampling error between sampling gradient and true gradient, and improve RBM network training precision.关键词
深度学习/受限玻尔兹曼机/采样算法/马尔科夫理论/并行回火/GFPTKey words
Deep learning/restricted Boltzmann machine (RBM)/sampling algorithm/Markov theory/parallel temper-ing/GFPT (Gradient fixing parallel tempering)引用本文复制引用
李飞,高晓光,万开方..基于改进并行回火算法的RBM网络训练研究[J].自动化学报,2017,43(5):753-764,12.基金项目
国家自然科学基金(61573285, 61305133), 中央高校基本科研业务费专项基金(3102015BJ(Ⅱ)GH01) 资助 Supported by National Natural Science Foundation of China(61573285, 61305133), Fundamental Research Funds for the Central Universities(3102015BJ(Ⅱ)GH01) （61573285, 61305133）
