摘要
大数据计算是当前云计算研究的热点之一.在电力信息化、精益化的建设过程中,业务复杂度不断提高,数据量与日俱增,这使得传统的数据加工性能日益劣化.在复杂的业务场景下,由于海量的电力数据,使得数据指标加工计算的效率非常低下,传统方式的加工任务经常耗时数个小时,难以满足用户的体验要求.为了解决这个问题,全面提升数据指标加工任务效率,基于对象化并行计算(Objectification Parallel Computing,OPC)框架实现了一种高效计算服务,OPC是分布式并行内存计算框架.在OPC框架中,大数据集被拆分成小数据集,并分布式地存储在集群内存中.OPC计算任务借鉴了分而治之和归并树的思想,将计算任务分成两个阶段:本地计算任务和计算结果收集汇总.计算任务基于本地计算数据进行计算,得到本地计算结果,然后将计算结果通过收集结点进行汇总合并,最后得到最终结果.OPC框架技术应用在国家电网公司工程生产管理系统(PMS)中,应用效果表明该技术稳定、可靠,性能提升几十至数百倍,可满足高效计算需求.
The big data computing is one of the researches focus in the Cloud Computing nowadays. With the development of electric power information, the business complexity continues to increase and the amount of data is increasing quickly which makes the traditional way of computing be worse and worse. This paper provides an effective computing of big data base on the Objectification Parallel Computing（OPC） to solve the above challenges. The small data set is split from Big Data, is distributedly stored in memory of OPC Cluster. In the effective compute server, making use of the thought of divide and rule and tree merging, there are two stages. The first stage is local data calculate. The intermediate calculation result can be obtained. The second stage is multistage summarizing. The final result can be returned to user. The solution has been applied to the power production management system（PMS） of State Grid of China. The results show that solution is efficiently reliable and meets users＇ requirement.
出处
《计算机系统应用》
2016年第9期92-97,共6页
Computer Systems & Applications
关键词
大数据
高效计算
对象化并行计算
分布式
big data
effective computing
objectification parallel
computing
distributed
