期刊文献+

一种优化分布式文件系统的文件合并策略 被引量:6

File merging strategy for optimization of distributed file system
下载PDF
导出
摘要 分布式文件系统的性能对整个分布式系统的性能有着重要的影响,以Hadoop分布式文件系统(HDFS)为研究目标,针对HDFS处理小文件数据性能差的问题,分析存在的问题,提出一种新的文件合并策略,优化系统I/O性能。实现结果表明这种合并策略能有效提高分布式文件系统的性能。 The performance of distributed file system has an important effect on the performance of the whole distributed system.This paper took Hadoop Distributed File System(HDFS) as the research object.In order to solve the poor performance problem of HDFS in term of processing small files,this paper analyzed the existing problem and presented a novel file merging strategy to optimize system I/O performance.The experiment results show this merging strategy can effectively improve the performance of distributed file system.
作者 陈剑 龚发根
出处 《计算机应用》 CSCD 北大核心 2011年第A02期161-163,共3页 journal of Computer Applications
关键词 分布式文件系统 HADOOP分布式文件系统 性能优化 distributed file system HDFS file system performance optimization
  • 相关文献

参考文献9

  • 1BOKHARI S, RUT1" B, WYCKOFF P, et al. Experimental analysis of a mass storage system [ J]. Concurrency and Computation: Practice and Experience, 2006, 18(4) : 1929-1950. 被引量:1
  • 2WANG FANG, YUE YINLIANG, FENF DAN, et al. High availability storage system based on two-level metadata management [ C]// FCST 2007: Proceedings of the 2007 Japan-China Joint Workshop on Frontier of Computer Science and Technology. Piscataway, N J: IEEE, 2007:41 -48. 被引量:1
  • 3LI HUAIYANG, LIU YAN, CAO QIANG. Approximate parameters analysis of a closed fork-join queue model in an object-based storage system [ C] // Proceedings of the Eighth International Symposium on Optical Storage and 2008 International Workshop on Information Data Storage, SPIE 7125. IS. 1. ] : SPIE, 2008:1 -6. 被引量:1
  • 4ZHAO TIEZHU, VERDI M, DONG SHOUBIN, et d. Evaluation of a performance model Lustre file system [ C]// Proceedings tff the fifth Annual ChinaGfid Conference. Piscataway, NJ: IIElZ.; 2010:. 191 -196. 被引量:1
  • 5ZHAO TIEZHU, HU JINLONG. Performance evaluation of parallel file system based on Lustre and grey theory [ C]//Proceedings of the 2010 Ninth International Conference on Grid and Cloud Computing. Washington, DC: IEEE Computer Society, 2010:118 -122. 被引量:1
  • 6赵铁柱,董守斌,Verdi MARCH,Simon SEE.面向并行文件系统的性能评估及相对预测模型[J].软件学报,2011,22(9):2206-2221. 被引量:7
  • 7KONSTANTIN S, HAIRONG K, SANJAY R, et al. The Hadoop distributed file system [ C]// Proceedings of the 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies. Piseataway, NJ: IEEE, 2010:1-10. 被引量:1
  • 8栾亚建,黄翀民,龚高晟,赵铁柱.Hadoop平台的性能优化研究[J].计算机工程,2010,36(14):262-263. 被引量:51
  • 9Apache Hadoop Project. SequenceFile Class [ EB/OL]. [ 2011-02-17 ]. http://hadoop, apache, org/common/docs/current/api/ org/apache/hadoop/io/SequenceFile, html. 被引量:1

二级参考文献9

共引文献55

同被引文献32

  • 1冯谷,高鹏.新型SQL注入技术研究与分析[J].计算机科学,2012,39(S3):415-417. 被引量:10
  • 2Wikipedia.CloudStorage.http://en.wikipedia.org/wiki/Cloud_ storage. 2012-5-9. 被引量:1
  • 3White T.周敏齐,王晓玲,金澈清,钱卫宁,译.Hadoop权威指南.北京:清华大学出版社,2010. 被引量:2
  • 4http://finance.21cn. com/stock/wmkzg/a/2014/0910/14/28200740. sht-ml. 被引量:1
  • 5www. zdnet. com. cn至顶网-石计算第一门户. 被引量:1
  • 6大数据架构hadoop http://blog. csdn. net/guoxiaoqian8028/article/details/18772363. 被引量:1
  • 7YU L, CHEN G, WANG W, et al. Msfss: A storage system for masssmall files [ C ] // Computer Supported Cooperative Work in Design,2007. CSCWD 2007. 11th International Conference on, [ S. 1.].IEEE, 2007:1087-1092. 被引量:1
  • 8BEAVERD, KUMAR S, LI H C,et al. Finding a Needle in Hay-stack ;Facebooks Photo Storage[C]//OSDI. 2010,10,Vancouver,BC : [ s. n. ] : 1-8. 被引量:1
  • 9TaobaoFile System 项目主页,http://tfs. taobao. org/. 被引量:1
  • 10LIUX, YU Q, LIAO J. FASTDFS: A High Performance DistributedFile System[ J] . ICIC express letters. Part B, Applications : an inter-national journal of research and surveys, 2014, 5(6) : 1741 - 1746. 被引量:1

引证文献6

二级引证文献22

相关作者

内容加载中请稍等...

相关机构

内容加载中请稍等...

相关主题

内容加载中请稍等...

浏览历史

内容加载中请稍等...
;
使用帮助 返回顶部