摘要
为在全比较(ATAC)问题的分布式计算中达到较高的性能,提出一种基于任务驱动启发式的数据分发策略,充分考虑分布式环境中的存储使用、数据本地性和负载平衡。提出两个分发数据的启发式规则;根据相关约束条件和任务需求,所有的数据项均可在本地进行处理,使对于所有计算任务均具备良好的数据本地性。实验结果表明,对于64个节点的集群,所提策略实现了80%的存储节约量和100%的数据本地性,获得了理想化线性加速的89%。所提策略在ATAC问题的分布式计算中达到了较高性能。
To achieve high performance in all-to-all comparison(ATAC)distributed computing,a task-driven heuristic data distribution strategy was proposed,which took full account of storage usage,data locality and load balancing in distributed environment.Two heuristic rules were proposed to distribute data.According to the relevant constraints and task requirements,all data items were processed locally,which made all computing tasks have good data locality.Experimental results show that for 64 nodes cluster,the proposed strategy achieves 80%storage savings and 100%data locality,and achieves 89%ideal linear accele-ration.Therefore,the proposed strategy achieves high performance in distributed computing of ATAC problem.
作者
余先昊
周凤
YU Xian-hao;ZHOU Feng(College of Computer and Information Engineering,Guizhou University of Commerce,Guiyang 550001,China;College of Computer Science and Technology,Guizhou University,Guiyang 550025,China)
出处
《计算机工程与设计》
北大核心
2022年第3期751-756,共6页
Computer Engineering and Design
基金
贵州省科技计划基金项目(黔科合基础[2017]1051)。
关键词
全比较
启发式
分布式计算
数据分发
数据本地性
all-to-all comparison
heuristic
distributed computing
data distribution
data locality
