摘要
随着移动通信技术的飞速发展,有限的频谱利用资源与大量频谱通信需求之间的矛盾也日益加剧,需要新的智能方法来提高频谱利用率。本文提出了一种基于分布式优先经验池结合双深度Q网络的多用户动态频谱接入方法。通过该方法,次用户可以在动态变化的认知无线网络环境下根据自己感知信息来不断地学习,选择空闲信道完成频谱接入任务来提高频谱利用率。该方法采用分布式强化学习框架,将每个次用户视为一个智能体,各个智能体采用标准单智能体强化学习方法进行学习以降低底层计算开销。另外,该方法在神经网络训练的基础上加入优先级采样,优化了神经网络的训练效率以帮助次用户选择出最优策略。仿真实验结果表明该方法能提高接入信道时的成功率、降低碰撞率和提升通信速率。
With the rapid development of mobile communication technology,the contradiction between the limited spectrum utilization resources and the demand of a lot of spectrum communication is increasingly aggravated.New intelligent methods are needed to improve the utilization rate of spectrum.A multi-user dynamic spectrum access method based on distributed priority experience pool and double deep Q network is proposed.This method can help the secondary users to continuously learn according to their perceived environment information in the dynamic environment,and choose the idle channel to complete the spectrum access task for improving the spectrum utilization rate.In this method,a distributed reinforcement learning framework is adopted,and each secondary user is regarded as an agent.Each agent learns by using standard single-agent reinforcement learning method to reduce the underlying computing overhead.In addition,the method adds priority sampling on the basis of neural network training,and then optimizes the training efficiency of neural network to help sub-users choose the optimal strategy.The simulation results show that this method can improve the success rate,reduce the collision rate and improve the communication rate.
作者
何一汕
王永华
万频
王磊
伍文韬
He Yi-shan;Wang Yong-hua;Wan Pin;Wang Lei;Wu Wen-tao(School of Automation,Guangdong University of Technology,Guangzhou 510006,China)
出处
《广东工业大学学报》
CAS
2023年第4期85-93,共9页
Journal of Guangdong University of Technology
基金
国家自然科学基金资助项目(61971147)。
关键词
动态频谱接入
分布式强化学习
优先经验池
深度强化学习
dynamic spectrum access
distributed reinforcement learning
prioritized experience pool
deep reinforcement learning
