摘要
利用Apache Sqoop将关系型数据转移到非结构化数据库(本文选用HBase)中,通过分析Sqoop并行处理的性能给出了一种关系模式到非结构化模式转换的一种方法,并尝试了使用相关性感知技术对Sqoop作业进行优化,结果表明,通过适当安排NoSQL的列族结构和使用相关性感知技术安排数据在各节点间的布局,可以大大提升数据转换的性能。
In this paper,Apache Sqoop is used to transfer relational data to unstructured database(HBase is selected in this paper).By analyzing the performance of Sqoop parallel processing,we give a method of transforming relational mode to unstructured mode,and try to optimize Sqoop jobs by using related awareness technology.The results show that by properly arranging the column family structure of NoSQL and using related awareness technology,it can greatly improve the performance of data conversion by arranging the layout of data among nodes.
作者
林徐
LIN Xu(School of Computers Engineering, Anhui Sanlian University, Hefei 230601,China;Key Laboratory of Traffic Information and Safety for Colleges and Universities in Anhui Province, Hefei 230601,China)
出处
《榆林学院学报》
2020年第6期69-72,共4页
Journal of Yulin University
基金
2019年安徽省教育厅高校自然科学重点项目(KJ2019A0903)。
