摘要
为了提高强化学习算法的运行效率和收敛速度,提出了一种基于路径引导知识启发的强化学习方法PHQL。采用PHQL方法,不需要提前植入先导知识,agent在每一轮学习过程中更新Q表的同时,各个状态的路径知识也自主地建立起来并逐步修正和优化。算法利用已经获得的路径知识来指导和加速agent以后的强化学习过程,以减少agent学习过程的盲目性。分析了PHQL算法的探索、利用和启发3种行为的执行概率以及行为选取方法,提出一种行为选择概率随时间渐变的算法。以一个路径搜索问题为实例,对PHQL方法进行了验证、分析并与几种相关的强化学习算法进行了性能对比。实验结果表明,作者提出的方法对学习过程具有明显的加速作用,收敛性能有了较大的提高。
In order to improve the efficiency and convergence speed of reinforcement learning algorithm,a method of heuristic reinforcement learning based on acquired path guiding knowledge（PHQL） was proposed.During the learning process using PHQL,embeded background knowledge was not needed to agent.While the agent updated the Q table in each episode,the path knowledge was also built,revised and optimized autonomously.After that,the lerning process was guided and accelerated by means of acquired path knowledge,which decreased the blindness of agent.In addition,three sorts of action selection methods of exploration,exploit and heuristic were analyzed,and also a practical method that action selection probilities changed over time was put forward.In a path planning environment,the PHQL was compared to the standard Q-learning and other relevant reinforcement learning algorithms.The experimental results showed that present methods accelerate the learning process obviously,and improve the convergence speed distinctly.
出处
《四川大学学报（工程科学版）》
EI
CAS
CSCD
北大核心
2012年第5期136-142,共7页
Journal of Sichuan University (Engineering Science Edition)
基金
国家自然科学基金资助项目(60971088)
国家自然科学基金资助项目(60571048)
关键词
PHQL
Q学习
强化学习
路径规划
知识启发
PHQL
Q-learning
reinforcement learning
path planning
knowledge heuristic approach
