摘要
针对Hadoop平台的节点资源优化问题,提出MapReduce参数优化策略。获取新作业执行时的资源占用特征值,计算其与作业特征库中作业的相对距离,选择相对距离最小作业的配置作为新作业的最优配置,如果获取失败,则以迭代方式获取新作业的最优配置并更新作业特征库。实验结果表明,与默认参数配置相比,该策略能够提高作业执行效率,缩短作业运行时间。
For solving the problem of node resources optimization in Hadoop platform,this paper proposes a MapReduce parameter optimization strategy.When a new job is submitted,it first gets feature value of resource utilization,and then calculates the relative distance with the jobs in the signature database.At last,it selects the configuration of the job with the minimum relative distance as the optimal configuration.If the configuration is not found,it gets optimal configuration by the way of iteration and then updates the feature database.Experimental results show that the proposed strategy can effectively improve the efficiency of job execution and reduce the execution time compared with the default parameter configuration.
出处
《计算机工程》
CAS
CSCD
北大核心
2016年第1期1-6,共6页
Computer Engineering
基金
国家自然科学基金资助项目(61272447)
国家科技支撑计划基金资助项目(2012BAH18B05)
