摘要
笔者在分析大数据相关开源框架和技术的基础上,提出基于开源框架实现边缘侧数据汇聚接入大数据云端、云端数据采集、分布式流处理、实时数据计算以及数据存储的框架设计与关键技术应用。该功能设计在某大型能源企业电力大数据系统中已成功实践应用,有效解决了实时数据采集应用中数据量大、响应速度快、数据传输可靠性要求高等问题。
Based on the analysis of open source frameworks and technologies related to large data,this paper proposes the framework design and key technology application of edge-side data aggregation and access to large data cloud,cloud data acquisition,distributed stream processing,real-time data calculation and data storage based on open source frameworks.The function design has been successfully applied in a large power data system of a large energy enterprise.It effectively solves the problems of large amount of data,fast response speed and high reliability of data transmission in the application of real-time data acquisition.
作者
李夏光
Li Xiaguang(Shenhua Helishi Information Technology Co., Ltd., Beijing 100001, China)
出处
《信息与电脑》
2019年第1期155-156,共2页
Information & Computer
关键词
大数据
实时数据采集
开源
电力
功能设计
big data
real-time data acquisition
open source
power
functional design
