摘要
围绕超算资源的易用性和多类软件的集成以及协作需求,开发了超算环境下的科学工作流应用平台,设计了异步并发的流程执行引擎,采取调度算法和调度器、引擎相分离的设计策略,给出了资源调度方案。提出了局部资源池化技术和资源预约算法,并比较分析了五种常用调度算法的性能,给出了算法选择的建议。实际应用表明设计的引擎能够支撑复杂工作流的灵活执行方式,给出的资源调度方案能够满足超算环境下工作流应用的高效执行。
In general,lots of software need to be cooperated together for particular object in scientific experiments and engineering domains.This paper described a new scientific workflow application platform in HPC environment.This platform contained an engine with high concurrency and asynchronous framework to process workflow application.Scheduler,planner and engine were decoupled from each other,which allowed that the three components could develop independently:scheduler for scheduling algorithms implementation,planner for collecting the scheduling information and engine for workflow driver.Scheduler and planner used resource advance reservation and local pooling mechanism to increase the performance of workflow execution.This paper also implemented and compared five scheduling algorithms as to their performance on testing graph set,and got some useful advices of algorithm selection.Real application shows that engine can support various execution strategies and resource scheduling solution can help increase efficiency of workflow execution in supercomputing environment.
作者
李于锋
莫则尧
肖永浩
赵士操
段博文
Li Yufeng;Mo Zeyao;Xiao Yonghao;Zhao Shicao;Duan Bowen(Institute of Computer Application,Chinese Academy of Engineering Physics,Mianyang Sichuan 621900,China;Institute of Applied Physics & Computational Mathematics,Beijing 100094,China)
出处
《计算机应用研究》
CSCD
北大核心
2019年第6期1723-1726,1736,共5页
Application Research of Computers
基金
国家重点研发计划资助项目(2016YFB0201504,2018YFB0703903)
关键词
科学工作流
高性能计算
资源调度
工作流引擎
scientific workflow
high performance computing
resource scheduling
workflow engine
