摘要
针对现有研究寻找数据流最佳排序这一过程本身花费时间过多,没有工程可行的寻找最佳排序的有效方法的问题,提出基于深度强化学习的流排序和调度框架PSNDRL.该框架包括3个关键模块,即创建时间触发(Time-Triggered,TT)流之间关系图的预处理模块、挖掘和量化TT流之间复杂的相关关系并选择概率值最高的TT流的代理模块、进行TT流调度和奖励计算的环境模块,利用图卷积网络和强化学习从大量的TT流中智能探索流特征以及流之间的复杂相关关系对基于可满足性模理论(Satisfiability Modulo Theories,SMT)流调度算法求解时间的潜在影响.通过训练该框架,学习得到一个高效率的TT流策略排序网络,用于在利用SMT流调度算法调度TT流时进行TT流选择.通过与随机排序和基线排序方法进行对比,验证PSNDRL的有效性.结果表明:与随机排序方法的总调度时间、基线排序方法的最大总调度时间相比,PSNDRL的总调度时间分别减少了24.63%和25.95%.所提框架为提高时间敏感网络(Time Sensitive Networking,TSN)流调度效率的研究提供了新的方向.
Addressing the time-consuming process of finding the optimal data flow sorting in existing research and the lack of an effective engineering method to achieve this,this study proposes the PSNDRL framework for flow sorting and scheduling based on deep reinforcement learning.This framework comprises three key modules:the Preprocessing module for establishing a relationship map between Time-Triggered(TT)flows,the Agent module for identifying and quantifying complex cor-relations between TT flows and selecting the flow with the highest probability value,and the Environ-ment module for TT flow scheduling and reward calculation.By leveraging graph convolutional net-works and reinforcement learning,the PSNDRL framework intelligently explores the impact of flow characteristics and intricate relationships between flows from numerous TT flows on the solving time of flow scheduling algorithms based on Satisfiability Modulo Theories(SMT).Through the training of this framework,an efficient TT flow strategy sorting network is developed for TT flow selection dur-ing scheduling with SMT algorithms.To validate the effectiveness of PSNDRL,this study compares it with random sorting and baseline sorting methods.Results demonstrate that the total scheduling time of PSNDRL is reduced by 25.95%and 24.62%,respectively,compared to random sorting and baseline sorting methods.This framework offers a novel direction for research to improve the effi-ciency of Time Sensitive Networking(TSN)flow scheduling.
作者
邓金雪
李纯喜
李宗辉
赵永祥
DENG Jinxue;LI Chunxi;LI Zonghui;ZHAO Yongxiang(School of Electronics and Information Engineering,Beijing Jiaotong University,Beijing 100044,China;School of Computer and Information Technology,Beijing Jiaotong University,Beijing 100044,China)
出处
《北京交通大学学报》
CAS
CSCD
北大核心
2024年第2期144-153,共10页
JOURNAL OF BEIJING JIAOTONG UNIVERSITY
基金
国家重点研发计划(2022YFB3303700)。
关键词
时间敏感网络
深度强化学习
流排序
流调度
time sensitive networking
deep reinforcement learning
flow sorting
flow scheduling
