摘要
Hadoop平台作为一个开源的在集群上运行大型数据库处理的框架受到了各个公司的青睐,然而要在Hadoop集群上运行一个作业必须手动设置将近200多个复杂的参数,如何设置这些参数对普通用户来说是非常困难的,该文针对这个问题提出了一种基于策略选择的抽样算法,通过在Hadoop中加入策略感知层,实验结果表明改进的Hadoop框架可以自动优化设置这些复杂的参数,从而提高整个系统的运行效率。
Hadoop platform as an open source cluster framework for running large-scale database processing by each company's favor,how.ever,run job on a Hadoop cluster must be set manually almost morethan 200 parameters,how to set these parameters for the ordinaryus.er is very diffic-ult.this paper proposes a solution for this problem,adding Hadoop Strategies perceived layer,that is use a sampling algo.rithm based strategic choice.the experimental results show that improved the Hadoop framework can automatically optimize the set.tings of these parameters,thereby improving the operating efficiency of the entire system.
出处
《电脑知识与技术》
2012年第4X期2768-2772,共5页
Computer Knowledge and Technology
