不确定性和隐状态是目前强化学习所要面对的重要难题.本文提出了一种新的算法MA-Q-learning算法来求解带有这种不确定性的POMDP问题近似最优策略.利用M em etic算法来进化策略,而Q学习算法得到预测奖励来指出进化策略的适应度值.针对隐...不确定性和隐状态是目前强化学习所要面对的重要难题.本文提出了一种新的算法MA-Q-learning算法来求解带有这种不确定性的POMDP问题近似最优策略.利用M em etic算法来进化策略,而Q学习算法得到预测奖励来指出进化策略的适应度值.针对隐状态问题,通过记忆agent最近经历的确定性的有限步历史信息,与表示所有可能状态上的概率分布的信度状态相结合,共同决策当前的最优策略.利用一种混合搜索方法来提高搜索效率,其中调整因子被用于保持种群的多样性,并且指导组合式交叉操作与变异操作.在POMDP的Benchm ark实例上的实验结果证明本文提出的算法性能优于其他的POMDP近似算法.展开更多
The problem of generating optimal paths for curvature-constrained unmanned aerial vehicles （UAVs） performing surveillance of multiple ground targets is addressed in this paper. UAVs are modeled as Dubins vehicles so...The problem of generating optimal paths for curvature-constrained unmanned aerial vehicles （UAVs） performing surveillance of multiple ground targets is addressed in this paper. UAVs are modeled as Dubins vehicles so that the constraints of UAVs＇ minimal turning radius can be taken into account. In view of the effective surveillance range of the sensors equipped on UAVs, the problem is formulated as a Dubins traveling salesman problem with neighborhood （DTSPN）. Considering its prohibitively high computational complexity, the Dubins paths in the sense of terminal heading relaxation are introduced to simplify the calculation of the Dubins distance, and a boundary-based encoding scheme is proposed to determine the visiting point of every target neighborhood. Then, an evolutionary algorithm is used to derive the optimal Dubins tour. To further enhance the quality of the solutions, a local search strategy based on approximate gradient is employed to improve the visiting points of target neighborhoods. Finally, by a minor modification to the individual encoding, the algorithm is easily extended to deal with other two more sophisticated DTSPN variants （multi-UAV scenario and multiple groups of targets scenario）. The performance of the algorithm is demonstrated through comparative experiments with other two state-of-the-art DTSPN algorithms identified in literature. Numerical simulations exhibit that the algorithm proposed in this paper can find high-quality solutions to the DTSPN with lower computational cost and produce significantly improved performance over the other algorithms.展开更多
文摘不确定性和隐状态是目前强化学习所要面对的重要难题.本文提出了一种新的算法MA-Q-learning算法来求解带有这种不确定性的POMDP问题近似最优策略.利用M em etic算法来进化策略,而Q学习算法得到预测奖励来指出进化策略的适应度值.针对隐状态问题,通过记忆agent最近经历的确定性的有限步历史信息,与表示所有可能状态上的概率分布的信度状态相结合,共同决策当前的最优策略.利用一种混合搜索方法来提高搜索效率,其中调整因子被用于保持种群的多样性,并且指导组合式交叉操作与变异操作.在POMDP的Benchm ark实例上的实验结果证明本文提出的算法性能优于其他的POMDP近似算法.
基金co-supported by the Foundation for Innovative Research Groups of the National Natural Science Foundation of China (No. 61321002)the Projects of Major International (Regional) Joint Research Program NSFC (No. 61120106010)+1 种基金Beijing Education Committee Cooperation Building Foundation Project, the National Natural Science Foundation of China (No. 61304215)Beijing Outstanding Ph.D. Program Mentor (No. 20131000704)
文摘The problem of generating optimal paths for curvature-constrained unmanned aerial vehicles （UAVs） performing surveillance of multiple ground targets is addressed in this paper. UAVs are modeled as Dubins vehicles so that the constraints of UAVs＇ minimal turning radius can be taken into account. In view of the effective surveillance range of the sensors equipped on UAVs, the problem is formulated as a Dubins traveling salesman problem with neighborhood （DTSPN）. Considering its prohibitively high computational complexity, the Dubins paths in the sense of terminal heading relaxation are introduced to simplify the calculation of the Dubins distance, and a boundary-based encoding scheme is proposed to determine the visiting point of every target neighborhood. Then, an evolutionary algorithm is used to derive the optimal Dubins tour. To further enhance the quality of the solutions, a local search strategy based on approximate gradient is employed to improve the visiting points of target neighborhoods. Finally, by a minor modification to the individual encoding, the algorithm is easily extended to deal with other two more sophisticated DTSPN variants （multi-UAV scenario and multiple groups of targets scenario）. The performance of the algorithm is demonstrated through comparative experiments with other two state-of-the-art DTSPN algorithms identified in literature. Numerical simulations exhibit that the algorithm proposed in this paper can find high-quality solutions to the DTSPN with lower computational cost and produce significantly improved performance over the other algorithms.
