期刊文献+

异构云环境下MapReduce高效性的优化研究 被引量:2

Research on High Efficiency Optimization of MapReduce in Heterogeneous Cloud Environment
下载PDF
导出
摘要 为提高MapReduce在异构云计算环境下的执行效率,提出一种异构环境下基于负载类型的自适应调度算法——adaptive scheduler based on workload type in heterogeneous environment(ASBT-HE),通过反馈运行结果到Job Tracker,将后续任务根据负载类型分配到相应的Task Trackers上。但为满足用户的Qo S需求、更好的平衡系统负载,将该算进一步法优化为Better ASBT-HE(BASBT-HE)。BASBT-HE能够在考虑Qo S约束的基础上根据负载情况自动调整ASBT-HE中队列的长度。仿真实验结果表明优化后的MapReduce在异构云计算环境下具有高效性,并且考虑了用户Qo S需求,简化了系统参数配置。 To improve the efficiency of MapReduce in heterogeneous cloud computing environment, an adaptive scheduler is presented based on workload type in heterogeneous environment (ASBT-HE). It feeds back results to the JobTracker to assign subsequent tasks to corresponding TaskTrackers. But in order to meet user' s QoS require- ments and better balance the loads of system, ASBT-I-IE is optimized a better ASBT-HE (BASBT-HE). On the ba- sis of considering the QoS constraints, BASBT-HE can adjust the length of queues in ASBT-HE automatically ac- cording to the actual loads. The ruslts of simulation experiments show that the optimized MapReduce has high effi- ciency in heterogeneous cloud computing environment. And it considers the user' s QoS requirements and simplifies the system configuration.
出处 《科学技术与工程》 北大核心 2014年第31期73-77,共5页 Science Technology and Engineering
基金 河北省高等学校科学技术研究重点项目(ZD2014054) 河北省重点基础研究项目(14964206D)资助
关键词 云计算 MapRedcue 调度算法 高效性 优化 cloud computing MapRedcue scheduler high efficience optimization
  • 相关文献

参考文献15

  • 1Gong Y D,Ying Z Q,Lin M H.A survey of cloud computing.Proceedings of the 2nd International Conference on Green Communications and Networks.Berlin:Springer Berlin Heidelberg,2013:79-81. 被引量:1
  • 2陈波,张曦煌.基于分层与容错机制的云计算负载均衡策略[J].计算机应用,2013,33(11):3155-3159. 被引量:8
  • 3柳敬..云计算平台的成本效用研究[D].北京邮电大学,2010:
  • 4刘鹏主编..实战Hadoop[M].北京:电子工业出版社,2011:456.
  • 5Dean J,Ghemawat S.MapReduce:simplified data processing on large clusters.Communications of the ACM,2008 ;51 (1):107-113. 被引量:1
  • 6程苗.基于云计算的Web数据挖掘[J].计算机科学,2011,38(B10):146-149. 被引量:51
  • 7Kang U,Tsourakakis C E,Faloutsos C.Pegasus:a peta-scale graph mining system-implementation and observations.Proc of IEEE International Conference on Data Mining.Piscataway:IEEE,2009:229-238. 被引量:1
  • 8李建锋,彭舰.云计算环境下基于改进遗传算法的任务调度算法[J].计算机应用,2011,31(1):184-186. 被引量:203
  • 9Kwon Y,Balazinska M,Howe B,et al.Skewtune:mitigating skew in MapReduce application.Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data.New York:ACM,2012:25-36. 被引量:1
  • 10Zaharia M,Konwinski A,Joseph A D,et al.Improving MapReduce performance in heterogeneous environments.Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation.Berkeley,USA:USENIX Association,2008:29-42. 被引量:1

二级参考文献35

  • 1席景科,闫大顺.Web数据挖掘中数据集成问题的研究[J].计算机工程与设计,2006,27(8):1366-1368. 被引量:6
  • 2米勒.云计算[M].史美林,姜进磊,孙瑞志,等译.北京:机械工业出版社,2009:125-128. 被引量:13
  • 3FOSTER I, YONG ZHAO, RAICU I, et al. Cloud computing and grid computing 360-degree compared[C] // Proceedings of the 2008 Grid Computing Environments Workshop. Washington, DC: IEEE Computer Society, 2008:1 - 10. 被引量:1
  • 4ARMBRUST M, FOX A, GRIFFITH R, et al. Above the clouds: A Berkeley view of cloud eomputing[EB/OL]. [2010 -01 -25]. http://www, eecs. berkeley, edu/Pubs/TechRpts/20Og/EECS-20og- 28. pdf. 被引量:1
  • 5BARROSO L A, DEAN J, HOLZLE U. Web search for a planet: the google cluster architecture[J]. IEEE Micro, 2003, 23(2) : 22 - 28. 被引量:1
  • 6CHIEN A, CALDER B, ELBERT S, et al. Entropia: Architecture and performance of an enterprise desktop grid system[J]. Journal of Parallel and Distributed Computing, 2003, 63(5):597-610. 被引量:1
  • 7KIM J S, NAM B, MARSH M, et al. Creating a robust desktop grid using peer-to-peer services[EB/OL]. [ 2009 - 10 - 16]. ftp://ftp. cs. umd. edu/pub/hpsl/papers/papers-pdf/ngs07.pdf. 被引量:1
  • 8ABRAHAM A, BUYYA R, NATH B. Nature's heuristics for scheduling jobs on computational grids[ C]// The 8th International Conference on Advanced Computing and Communications. New Delhi: Tata McGraw-Hill Publishing, 2000:45-52. 被引量:1
  • 9DEAN J, GHEMAWAT S. MapReduce: simplified data processing on large clusters[ C]//Proceedings of the 6th Symposium on Operating System Design and Implementation. New York: ACM, 2004:137 - 150. 被引量:1
  • 10The CLOUDS Lab. Gridsim[ EB/OL]. [ 2010 - 06 - 25]. http:// www. cloudbus. org/gridsim/. 被引量:1

共引文献259

同被引文献10

引证文献2

二级引证文献6

相关作者

内容加载中请稍等...

相关机构

内容加载中请稍等...

相关主题

内容加载中请稍等...

浏览历史

内容加载中请稍等...
;
使用帮助 返回顶部