DQN等深度强化学习方法的学习过程与工作机制不透明,无法感知其决策依据与决策可靠性,使模型做出的决策饱受质疑,极大限制了深度强化学习的应用场景。为了解释智能体的决策机理,提出一种基于梯度的显著性图生成算法(saliency map genera...DQN等深度强化学习方法的学习过程与工作机制不透明,无法感知其决策依据与决策可靠性,使模型做出的决策饱受质疑,极大限制了深度强化学习的应用场景。为了解释智能体的决策机理,提出一种基于梯度的显著性图生成算法(saliency map generation algorithm based on gradient,SMGG)。使用高层卷积层生成的特征图梯度信息计算不同特征图的重要性,在模型的结构和内部参数已知的情况下,从模型最后一层入手,通过对特征图梯度的计算,生成不同特征图相对于显著性图的权重;对特征重要性进行正向和负向分类,利用有正向影响的权值将特征图中捕获的特征进行加权,构成当前决策的正向解释;利用对其他类别有负向影响的权值将特征图中捕获的特征进行加权,构成当前决策的反向解释。二者共同生成决策的显著性图,得出智能体决策行为的依据,实验证明了该方法的有效性。展开更多
目的:比较容积CT剂量指数(volume CT dose index,CTDI_(VOL))及基于水当量直径的体型特异性剂量估计值(size-specific dose estimate based on water equivalent diameter,SSDE_(WED))在衡量儿童头颅CT辐射剂量中的差异性,并分析CTDI_(V...目的:比较容积CT剂量指数(volume CT dose index,CTDI_(VOL))及基于水当量直径的体型特异性剂量估计值(size-specific dose estimate based on water equivalent diameter,SSDE_(WED))在衡量儿童头颅CT辐射剂量中的差异性,并分析CTDI_(VOL)、SSDE_(WED)与曝光量、水当量直径(water equivalent diameter,WED)的相关性,以为临床检查中儿童头颅CT辐射剂量衡量提供参考。方法:回顾性分析2021年1—12月于某院进行头颅CT检查的1297例患儿的临床资料,根据年龄将患儿分为≤1个月组、>1个月~4岁组、>4~10岁组、>10~15岁组。记录患儿的曝光量、年龄、CTDI_(VOL)、剂量长度乘积,并计算WED、转换因子及SSDE_(WED)。比较CTDI_(VOL)与SSDE_(WED)的差异;建立CTDI_(VOL)、SSDE_(WED)与曝光量、WED的回归模型,并采用Pearson分析CTDI_(VOL)、SSDE_(WED)与曝光量、WED之间的相关性;对比国内诊断参考水平(diagnostic reference level,DRL)、欧盟DRL及本医疗机构诊断参考水平(local diagnostic reference level,LDRL)的差异。采用SPSS 25.0统计学软件进行分析。结果:患儿头颅CT的CTDI_(VOL)为(9.22±1.63)mGy,SSDE_(WED)为(8.14±0.84)mGy,CTDI_(VOL)较SSDE_(WED)高13.27%,差异有统计学意义(t=47.66,P<0.001)。CTDI_(VOL)、SSDE_(WED)与曝光量、WED均呈正相关关系(P<0.001);CTDI_(VOL)、SSDE_(WED)与曝光量、WED回归模型拟合性较强(R^(2)为0.58~0.99)。与国内DRL及欧盟DRL比较,LDRL均处于较低水平。结论:在儿童头颅CT辐射剂量衡量中,相较于CTDI_(VOL),SSDE_(WED)对辐射剂量的衡量更准确。同时定期对医疗机构的DRL值进行统计更新并优化检查参数,是减少辐射剂量的重要方式。展开更多
For Future networks, many research projects have proposed different architectures around the globe;Software Defined Network(SDN) architectures, through separating Data and Control Layers, offer a crucial structure for...For Future networks, many research projects have proposed different architectures around the globe;Software Defined Network(SDN) architectures, through separating Data and Control Layers, offer a crucial structure for it. With a worldwide view and centralized Control, the SDN network provides flexible and reliable network management that improves network throughput and increases link utilization. In addition, it supports an innovative flow scheduling system to help advance Traffic Engineering(TE). For Medium and large-scale networks migrating directly from a legacy network to an SDN Network seems more complicated & even impossible, as there are High potential challenges, including technical, financial, security, shortage of standards, and quality of service degradation challenges. These challenges cause the birth and pave the ground for Hybrid SDN networks, where SDN devices coexist with traditional network devices. This study explores a Hybrid SDN network’s Traffic Engineering and Quality of Services Issues. Quality of service is described by network characteristics such as latency, jitter, loss, bandwidth,and network link utilization, using industry standards and mechanisms in a Hybrid SDN Network. We have organized the related studies in a way that the Quality of Service may gain the most benefit from the concept of Hybrid SDN networks using different algorithms and mechanisms: Deep Reinforcement Learning(DRL), Heuristic algorithm, K path partition algorithm, Genetic algorithm, SOTE algorithm, ROAR method, and Routing Optimization with different optimization mechanisms that help to ensure high-quality performance in a Hybrid SDN Network.展开更多
The emergence of beyond 5G networks has the potential for seamless and intelligent connectivity on a global scale.Network slicing is crucial in delivering services for different,demanding vertical applications in this...The emergence of beyond 5G networks has the potential for seamless and intelligent connectivity on a global scale.Network slicing is crucial in delivering services for different,demanding vertical applications in this context.Next-generation applications have time-sensitive requirements and depend on the most efficient routing path to ensure packets reach their intended destinations.However,the existing IP(Internet Protocol)over a multi-domain network faces challenges in enforcing network slicing due to minimal collaboration and information sharing among network operators.Conventional inter-domain routing methods,like Border Gateway Protocol(BGP),cannot make routing decisions based on performance,which frequently results in traffic flowing across congested paths that are never optimal.To address these issues,we propose CoopAI-Route,a multi-agent cooperative deep reinforcement learning(DRL)system utilizing hierarchical software-defined networks(SDN).This framework enforces network slicing in multi-domain networks and cooperative communication with various administrators to find performance-based routes in intra-and inter-domain.CoopAI-Route employs the Distributed Global Topology(DGT)algorithm to define inter-domain Quality of Service(QoS)paths.CoopAI-Route uses a DRL agent with a message-passing multi-agent Twin-Delayed Deep Deterministic Policy Gradient method to ensure optimal end-to-end routes adapted to the specific requirements of network slicing applications.Our evaluation demonstrates CoopAI-Route’s commendable performance in scalability,link failure handling,and adaptability to evolving topologies compared to state-of-the-art methods.展开更多
文摘DQN等深度强化学习方法的学习过程与工作机制不透明,无法感知其决策依据与决策可靠性,使模型做出的决策饱受质疑,极大限制了深度强化学习的应用场景。为了解释智能体的决策机理,提出一种基于梯度的显著性图生成算法(saliency map generation algorithm based on gradient,SMGG)。使用高层卷积层生成的特征图梯度信息计算不同特征图的重要性,在模型的结构和内部参数已知的情况下,从模型最后一层入手,通过对特征图梯度的计算,生成不同特征图相对于显著性图的权重;对特征重要性进行正向和负向分类,利用有正向影响的权值将特征图中捕获的特征进行加权,构成当前决策的正向解释;利用对其他类别有负向影响的权值将特征图中捕获的特征进行加权,构成当前决策的反向解释。二者共同生成决策的显著性图,得出智能体决策行为的依据,实验证明了该方法的有效性。
文摘For Future networks, many research projects have proposed different architectures around the globe;Software Defined Network(SDN) architectures, through separating Data and Control Layers, offer a crucial structure for it. With a worldwide view and centralized Control, the SDN network provides flexible and reliable network management that improves network throughput and increases link utilization. In addition, it supports an innovative flow scheduling system to help advance Traffic Engineering(TE). For Medium and large-scale networks migrating directly from a legacy network to an SDN Network seems more complicated & even impossible, as there are High potential challenges, including technical, financial, security, shortage of standards, and quality of service degradation challenges. These challenges cause the birth and pave the ground for Hybrid SDN networks, where SDN devices coexist with traditional network devices. This study explores a Hybrid SDN network’s Traffic Engineering and Quality of Services Issues. Quality of service is described by network characteristics such as latency, jitter, loss, bandwidth,and network link utilization, using industry standards and mechanisms in a Hybrid SDN Network. We have organized the related studies in a way that the Quality of Service may gain the most benefit from the concept of Hybrid SDN networks using different algorithms and mechanisms: Deep Reinforcement Learning(DRL), Heuristic algorithm, K path partition algorithm, Genetic algorithm, SOTE algorithm, ROAR method, and Routing Optimization with different optimization mechanisms that help to ensure high-quality performance in a Hybrid SDN Network.
文摘The emergence of beyond 5G networks has the potential for seamless and intelligent connectivity on a global scale.Network slicing is crucial in delivering services for different,demanding vertical applications in this context.Next-generation applications have time-sensitive requirements and depend on the most efficient routing path to ensure packets reach their intended destinations.However,the existing IP(Internet Protocol)over a multi-domain network faces challenges in enforcing network slicing due to minimal collaboration and information sharing among network operators.Conventional inter-domain routing methods,like Border Gateway Protocol(BGP),cannot make routing decisions based on performance,which frequently results in traffic flowing across congested paths that are never optimal.To address these issues,we propose CoopAI-Route,a multi-agent cooperative deep reinforcement learning(DRL)system utilizing hierarchical software-defined networks(SDN).This framework enforces network slicing in multi-domain networks and cooperative communication with various administrators to find performance-based routes in intra-and inter-domain.CoopAI-Route employs the Distributed Global Topology(DGT)algorithm to define inter-domain Quality of Service(QoS)paths.CoopAI-Route uses a DRL agent with a message-passing multi-agent Twin-Delayed Deep Deterministic Policy Gradient method to ensure optimal end-to-end routes adapted to the specific requirements of network slicing applications.Our evaluation demonstrates CoopAI-Route’s commendable performance in scalability,link failure handling,and adaptability to evolving topologies compared to state-of-the-art methods.
