摘要
分布式处理平台作为大数据技术重要组成部分,其低效率、高能耗问题不容忽视.针对这一问题,本文通过将现有大规模的数据处理节能算法划分为流式数据处理、批量数据处理、图数据处理以及彼此交互的数据处理四种节能算法进行分析探讨,其中彼此交互的数据处理节能算法又可划分为偏向批的交互数据处理、偏向流的交互数据处理以及偏向图的交互数据处理三种节能算法,并进行综合的讨论分析,讨论了现有分布式处理架构与节能算法存在的一系列问题(如对集群服务质量的影响、对集群性能的影响等).最后,对适应节能的分布式处理体系结构、节能计算与集群数据处理的适应性、节能计算与集群数据处理的普适性、集群执行节能算法的QoS约束保证以及集群执行节能算法的性能质量保证五个方面进行了展望分析.
Distributed processing platform,which is one of the significant parts of big data technology,has to face the problems of low efficiency and high energy consumption.To this point,the existing energy-efficient algorithms for large-scale data processing are analyzed and discussed in this paper,which can be divided into four kinds:stream data processing,batch data processing,graph data processing and interactive data processing.Moreover,the energy-efficient algorithms for interactive data processing can be further divided into three types:the interactive data processing with emphasis on batch,the interactive data processing with emphasis on stream and the interactive data processing with emphasis on graph.With the analysis of above algorithms,a series of problems in the existing distributed processing architecture and energyefficient algorithms are discussed,such as the influence on QoS and performance of the cluster.Finally, five aspects are prospected and analyzed,including the distributed computing architecture which adapts to energy efficiency,the adaptability of energy-efficient computing and data processing,the universality of energy-efficient computing and data processing,the QoS constraint guarantee for execution of energy-efficient algorithms and the performance quality assurance when executing these algorithms.
作者
于炯
蒲勇霖
鲁亮
刘粟
YU Jiong;PU Yonglin;LU Liang;LIU Su(School of Information Science and Engzneemng,Xinfiang University,Urumqi Xinjiang 830046,China;School of Computer Science and Technology,Civil Aviation University of China,Tianjin 300300,China)
出处
《新疆大学学报（自然科学版）》
CAS
2018年第4期389-401,共13页
Journal of Xinjiang University(Natural Science Edition)
基金
国家自然科学基金项目(61262088
61462079
61562086
61363083
61562078)
国家科技部科技支撑项目(2015BAH02F01)
新疆研究生科研创新项目(XJGRI2016028)
关键词
分布式处理
大数据技术
绿色计算
能耗效益
服务质量
distributed computing
big data technology
green computing
energy-efficient
quality of service
