摘要
针对B5G无线接入网网络切片中多样化的服务需求和基站数量增加所导致的用户需求感知速度慢,计算和存储压力大等问题,提出了一种基于分布式多智能体鲁棒近端策略优化的多基站需求感知网络切片架构。此架构由参数服务器,样本缓存和多个鲁棒近端策略优化(RobustProximalPolicyOptimization,RPPO)智能体组成。其中每个基站都有一个RPPO智能体和样本缓存,前者在近端策略优化算法的基础上引入优势值归一化,策略熵和价值裁剪优化机制来探索高效的资源管理方案,后者存储用户样本数据。参数服务器中负责从样本缓存中抽取数据更新参数并发布到每个基站的RPPO模型中,基站中的RPPO智能体使用发布的参数实现每个基站的资源分配。仿真结果表明,所提架构在频谱效率,服务水平协议满意率,系统效用和用户需求感知速度等方面都有所提升。
A multi-base station demand-aware network slicing architecture based on Distributed Multi-Intelligent Robust Proximal Policy Optimization(DMRPPO)is proposed to address the problems of slow user demand sensing and high computational and storage pressure caused by diverse service demands and the increase in the number of base stations in B5G network slicing.This architecture consists of a parameter server,a sample cache,and multiple Robust Proximal Policy Optimization(RPPO)agent.In which each base station has an RPPO and a sample cache,the former introduces dominant value normalization,policy entropy and value trimming optimization mechanisms based on the PPO to explore efficient resource management schemes,and the latter stores user sample data.The parameter server is responsible for extracting data from the sample cache to update the parameters and publish them to the RPPO,and the agent in the base station uses the published parameters to realize the resource allocation of each base station.Simulation results show that the proposed improves spectral efficiency,service level agreement satisfaction rate,system utility and user demand awareness speed.
作者
孙晓川
秦贞滕
张琪
崔东艳
黄天宇
SUN Xiaochuan;QIN Zhenteng;ZHANG Qi;CUI Dongyan;HUANG Tianyu(College of Artificial Intelligence,North China University of Science and Technology,Tangshan 063210,China;Hebei Key Laboratory of Industrial Perception,Tangshan 063210,China)
出处
《微电子学与计算机》
2024年第10期131-140,共10页
Microelectronics & Computer
基金
河北省教育厅科技项目(ZD2021088)。
关键词
网络切片
分布式架构
深度强化学习
资源管理
系统效用
network slicing
distributed architecture
deep reinforcement learning
resource management
system utility
