通信学报Issue(10)：92-99,8.DOI:10.3969/j.issn.1000-436x.2013.10.011
基于两层模糊划分的时间差分算法
TD algorithm based on double-layer fuzzy partitioning
摘要
Abstract
When dealing with the continuous space problems, the traditional Q-iteration algorithms based on lookup-table or function approximation converge slowly and are difficult to get a continuous policy. To overcome the above weak-nesses, an on-policy TD algorithm named DFP-OPTD was proposed based on double-layer fuzzy partitioning and its convergence was proved. The first layer of fuzzy partitioning was applied for state space, the second layer of fuzzy parti-tioning was applied for action space, and Q-value functions were computed by the combination of the two layer fuzzy partitioning. Based on the Q-value function, the consequent parameters of fuzzy rules were updated by gradient descent method. Applying DFP-OPTD on two classical reinforcement learning problems, experimental results show that the algo-rithm not only can be used to get a continuous action policy, but also has a better convergence performance.关键词
强化学习/在策略/梯度下降/两层模糊划分/连续行为策略Key words
reinforcement learning/on-policy/gradient descent/double layer fuzzy partitioning/continuous action policy分类
信息技术与安全科学引用本文复制引用
穆翔,刘全,傅启明,孙洪坤,周鑫..基于两层模糊划分的时间差分算法[J].通信学报,2013,(10):92-99,8.基金项目
国家自然科学基金资助项目(61070223,61103045,61070122,61272005)；江苏省自然科学基金资助项目(BK2012616)；江苏省高校自然科学研究基金资助项目(09KJA520002,09KJB520012)；吉林大学符号计算与知识工程教育部重点实验室基金资助项目(93K172012K04)Foundation Items：The National Natural Science Foundation of China(61070223,61103045,61070122,61272005) （61070223,61103045,61070122,61272005）
The Natural Science Foundation of Jiangsu Province(BK2012616) （BK2012616）
The High School Natural Foundation of Jiangsu Province(09KJA520002,09KJB520012) （09KJA520002,09KJB520012）
The Foundation of Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education of Jilin University(93K172012K04) （93K172012K04）
