摘要
针对强化学习收敛速度慢的问题,提出可在线更新的信息强度引导的启发式Q学习算法以加快强化学习算法的收敛速度。该算法在启发式强化学习算法的基础上引入依据每次训练回报情况进行在线更新的信息强度,通过结合强弱程度不同的动作信息强度更新的启发函数和状态—动作值函数来确定策略,从而提高算法收敛速度。给出该算法并对其收敛性进行证明,同时针对不同参数设置和仿真环境进行路径规划的仿真对比实验,得到实验结果:信息强度引导的启发式Q学习算法在成功率、达到目标位置所需步数及所获回报上均优于Q学习和基本启发式Q学习算法。实验结果表明,该算法能更快地得到回报较高的策略且不会陷入局部收敛,因而该算法能够有效提高算法的收敛速度。
Since reinforcement learning is time-consuming algorithm, this paper presented an online pheromone stringency guiding heuristically accelerated Q-learning algorithm to speed up the convergence rate. Based on the heuristically accelerated reinforcement learning, heuristic function added a pheromone stringency which could be updated online according to the rewards of the training, then combined with the value function to determine the policy. This paper proved the convergence of the algorithm. Meanwhile, simulation results show that pheromone stringency guiding heuristically accelerated Q-learning algorithm has better performance in the rate of success, steps to reach target location and rewards obtained in the learning. The algorithm can find the optimal policy faster and avoid getting into local convergence, thus effectively speeding up the convergence rate.
作者
吴昊霖
蔡乐才
高祥
Wu Haolin;Cai Lecai;Gao Xiang(School of Automation ＆ Information Engineering,Sichuan University of Science ＆ Engineering,Zigong Sichuan 643000,China;Artificial Intelligence Key Laboratory of Sichuan Province,Zigong Sichuan 643000,China;Yibin University,Yibin Sichuan 644000,China)
出处
《计算机应用研究》
CSCD
北大核心
2018年第8期2323-2327,共5页
Application Research of Computers
基金
国家自然科学基金资助项目(61202196)
人工智能四川省重点实验室项目(2015RYY02)
企业信息化与物联网测控技术四川省高校重点实验室开放基金项目(2014WZY01
2016WYJ02)
四川理工学院研究生创新基金资助项目(Y2016032)
关键词
强化学习
启发函数
信息强度
在线更新
收敛速度
reinforcement learning（RL）
heuristic function
pheromone stringency
online
convergence rate
