摘要
云计算和大数据已成为IT领域的研究热点,如何将云计算在数据存储和数据处理方面的优势应用于大数据领域具有重要的实际应用价值.开源的云平台OpenStack可方便地从硬件管理方面构建私有云,其存储模块Swift能够支持PB级的大数据存储.开源的云平台Hadoop在数据处理方面具有很强的优势,但在支持超大数据存储方面存在不足.通过对OpenStack中的存储模块Swift和Hadoop中的文件处理模块HDFS的比较分析,提出了将Swift和Hadoop的MapReduce技术结合来构建企业处理大数据的私有云计算系统方案.分析结果显示该方案是可行的,这种异构的私有云系统可以整合不同云计算平台各自的优势进行高效的大数据处理.
Cloud computing and big data has become a hotspot in the field of IT, and there is an important practical application value for applying cloud computing＇s advantages in data storage and data processing to the field of big data. OpenStack which is an open-source cloud platform could build a private cloud easily from the hardware management aspect, its Swift storage module can support PB-level data storage. Hadoop which is an open- source cloud platform has a strong advantage in data processing, but insufficient in support of large data storage. Through compare the object storage module swift in OpenStack with the file processing module HDFS in Hadoop, we put forward a scheme by combine Swift and Hadoop MapReduce technology which can build private cloud computing system for enterprise deal with big data. Analysis results show that the scheme is feasible,the heterogeneous private cloud system can integrate different cloud computing platforms＇ advantages for processing large data efficiently.
出处
《数学的实践与认识》
CSCD
北大核心
2014年第5期157-165,共9页
Mathematics in Practice and Theory
基金
国家自然科学基金(61070154
61373028)
中国博士后科学基金(20110490091)
中国博士科学基金特别资助(2013T60804)
