摘要
智能配用电数据具有海量以及多源异构等特点,针对这些数据需要实施有效的存储处理设计,以实现智能配用电数据的快速存储和处理,提出了基于Hadoop的智能配用电数据存储处理设计方案,建构智能配用电数据的资源、存储以及查询存储框架,实现对数据的存储资源管理、数据预处理以及数据归一化处理等,以有效实现海量数据的快速存储。在Top-k检索技术的应用下,实现数据的迅速查询。数据存储处理中通过密钥和密文管理分离,提升密钥管理的便利性,以此确保数据存储的安全性。通过对Hadoop智能配用电数据存储系统的实验分析,充分证实这一方法可以有效满足数据存储速度需求,提升数据安全,具有有效性以及可用性。
Intelligent power distribution data has the characteristics of massive and multi⁃source isomerism.In order to achieve the rapid storage and processing of intelligent power distribution data,it is needed to implement effective storage and processing design for these data.A Hadoop based intelligent power distribution data storage and processing design scheme is proposed,constructs the resources,storage and query storage framework of intelligent power distribution data,realizes the storage resource management,data preprocessing and data normalization processing of data,so as to effectively realize the rapid storage of massive data.And under the application of Top⁃k retrieval technology,the data can be queried quickly.In data storage processing,key and ciphertext management are separated to improve the convenience of key management,so as to ensure the security of data storage.Through the experimental analysis of Hadoop intelligent power distribution data storage system,it is fully proved that this method can effectively meet the requirements of data storage speed,improve data security,effectiveness and availability.
作者
曾麒杰
符晓洪
柳羿
何宽政
张升平
ZENG Qijie;FU Xiaohong;LIU Yi;HE Kuanzheng;ZHANG Shengping(Shenzhen Power Supply Bureau Co.,Ltd.,Shenzhen 518000,China)
出处
《电子设计工程》
2021年第11期136-139,共4页
Electronic Design Engineering
关键词
HADOOP
智能配用电
密钥管理
存储处理
Hadoop
intelligent power distribution
key management
storage processing
