期刊文献+

分布式文件系统元数据服务器高可用性设计 被引量:3

Design of Metadata High Availability in Distributed File Systems
下载PDF
导出
摘要 设计并实现了面向对象的分布式文件系统元数据服务器高可用方案,用于提高存储系统的可用性.系统使用集中式元数据管理服务器,通过日志文件和检查点文件对元数据进行保存;针对系统特点,该方案采用active/hot-standby模式实现元数据服务器冗余备份.对系统状态监控、日志及检查点数据同步复制、元数据服务器节点失败接管、防止系统split-brain等关键技术问题进行了深入研究和提出相应解决方法,并对影响系统恢复时间的因素进行了细致分析.测试表明,高可用功能的实现对系统性能影响可以随存储文件的增大而减少,并可在失败发生后的较短时间内完成主从服务器的切换. Design and implemented the Metadata High Availability in object-oriented Distributed File System to provide high availability to the whole storage system. The system uses a centralized metadata server and stores the metadata in log and checkpoint files. In the light of the system characters, redundant metadata server is used with active/hot-standby model. The key technical problems such as monitoring server status, replicating checkpoint and journal synchronously, failing over the failed server node and avoiding the occurrence of split-brain are deeply researched and corresponding solutions are provided. The factors that influence the recovery time are analyzed in detail. The test results showed that such distributed file systems can recovery from a metadata server failure in a short time with only a tiny performance influence when storing moderate size files.
出处 《小型微型计算机系统》 CSCD 北大核心 2013年第4期801-805,共5页 Journal of Chinese Computer Systems
基金 上海市科学技术委员会基金重点项目(10DZ1500200)资助
关键词 高可用 元数据 分布式文件系统 复制 失败接管 high availability metadata distributed file system replication fail over
  • 相关文献

参考文献12

  • 1Jardine R L, Basavaiah M, Krishnakuma K S. Method and apparatus for split-brain avoidance in a multi-processor system[ P]. U. S. Pa- tent 5991518,1999-11-23. 被引量:1
  • 2Shvachko K,Kuang H,Radia S. The hadoop distribuw.d file system [ C]. IEEE 26th Symposium on Mass Storage Systems and Tech- nologies, 2010. 被引量:1
  • 3MarzuUo K, Schmuck F. Supplying high availability with a standard network file system[ C]. Proceedings of the 8th International Con- ferenee on Distributed Computing Systems, Amsterdam, The Neth- erlands, IEEE Computer Society Press, 1988:447-453. 被引量:1
  • 4] Siegel A, Birman K, Marzullo K. Deceit: a flexible distributed file system [ C ]. Management of Replicated Data, Houston, Texas, USA. IEEE Computer Society Press, 1990:15-17. 被引量:1
  • 5Swart G, Birrel A, Hisgen A. Availability in the echo file system [ R/OL ]. Systems Research Center, Digital Equipment Corpora- tion,Tech Report: SRC-RR-112,http ://www. hpl. hp. com/techre- ports/Compaq-DEC/SRC-RR-112, hind, 1993 -09 -10/2011 10 -24. 被引量:1
  • 6Engelmann C ,Scott S L ,Leangsukun C ,et aL Symmetric active/ac- tive high availability for high-performance computing system serv- ices[ J]. Journal of Computers ,2006,1 ( 8 ) :43-54. 被引量:1
  • 7Schmuck F, Haskin R. GPFS: a shared-disk file system for large computing clusters[ C]. Proceedings of the Conference on File and Storage Technologies, Monterey, California, USA,2002:231-244. 被引量:1
  • 8Braam P J. Lustre: a scalable, high-performance file system C[ EB/ OL]. Cluster File Systems, Inc. ftp://ftp, uni-duisburg, de/linux/ filesys/Lustre/whitepaper, pdf,2003 -05 -09/2011-10-24. 被引量:1
  • 9Bhide A, Elnozahy E N, Morgan S P. A highly available network file server[ C]. Proceedings of the USENIX Conference, Dallas,TX, 1991 : 199-205. 被引量:1
  • 10Ghemawat S, Gobioff H, Leung S T. The google file system [ C ]. 19th ACM Symposium on Operating Systems Principles ,2003. 被引量:1

二级参考文献14

  • 1杨德志,黄华,张建刚,许鲁.大容量、高性能、高扩展能力的蓝鲸分布式文件系统[J].计算机研究与发展,2005,42(6):1028-1033. 被引量:28
  • 2黄华,张建刚,许鲁.蓝鲸分布式文件系统的分布式分层资源管理模型[J].计算机研究与发展,2005,42(6):1034-1038. 被引量:12
  • 3SIMS of UC Berkeley. How Much Information. http: ∥www. sims. berkeley. edu/how-much-info/, 2000-11-10 被引量:1
  • 4D. Anderson, J. Chase, A. Vahdat. Interposed request routing for scalable network storage. Duke University, Tech Rep: CS-2000-05, 2000 被引量:1
  • 5J. Menon, D. A. Pease, R. Rees, et al. IBM storage tank-A heterogeneous scalable SAN file system. IBM Systems Journal,2003, 42(2): 250~267 被引量:1
  • 6P. J. Braam. The lustre storage architecture. http: ∥www. lustre. org/docs/lustre. pdf, 2003-08 被引量:1
  • 7R. O. Weber. Information Technology-SCSI object-based storage device command. Technology Committee 10 Drafts. http:∥www. t10. org/ftp/t10/drafts/osd/osd-r10. pdf, 2004-07 被引量:1
  • 8J.S. Glider, C. F. Fuente, W. J. Scales. The software architecture of a SAN storage control system. IBM Systems Journal, 2003, 42(2): 232~249 被引量:1
  • 9Ellard D, Ledlie J, Malkani E et al. Passive NFS Tracing of Email and Research Workloads[C].Proc. of FAST'03. San Francisco, CA: [s. n.]. 2003. 被引量:1
  • 10Menon J, Pease D A, Rees R, et al. IBM Storage Tank-A Heterogeneous Scalable SAN File System[J]. IBM System Journal, 2003, 42(2): 250-267. 被引量:1

共引文献35

同被引文献19

  • 1SANJAY, GHEMAWAT, HOWARD GOBIOFF, SHUN- TAK LEUNG. The Google file system [ C ]. Proceedings of the nineteenth ACM symposium on Operating systems principles, 2003 : 19-22. 被引量:1
  • 2SHVACHKO K, KUANG H, RADIA S. The hadoop dis- tributed file system[ C]. IEEE 26th Symposium on Mass Storage Systems and Technologies, 2010 : 1-10. 被引量:1
  • 3RAO JUN, ROSS KENNETH A. Making B +-trees cacheconscious in main memory[ C]. Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data. Dallas, Texas, USA, 2000 : 475-486. 被引量:1
  • 4Marshall Kirk McKusick, Sean Quinlan. GFS; Evolution on fastforward [J]. ACMQueue, 2009, 7 (7).. 10-20. 被引量:1
  • 5Mohamad Sindi. Evaluating MPI implementations using HPL on an infiniband Nehalem Linux cluster [C] //Seventh Interna- tional Conference on Information Technology: New Genera- tions, 2010: 19-25. 被引量:1
  • 6Olson M. HAIXX)P: Scalable, flexible data storage and anal- ysis [J]. IQTQuarterly, 2010, 1 (3): 14-18. 被引量:1
  • 7Oracle. LustreTM 1.8 operations manuaL [EB/0L]. http:// wiki. lustre, org/images/0/09/821-0035_vl. 3. pdf, 2010. 被引量:1
  • 8Sun microsystems, lustre file system datasheet [R]. Santa Clara: Sun Microsystems, 2008. 被引量:1
  • 9Swapnil Patil, Garth Gibson. Scale and concurrency of GIGA +: File system directories with millions of files [C] //Pro- ceedings of the 9th USENIX Conference on File and Storage Technologies, 2011. 被引量:1
  • 10Cams P, Lang S, Ross R, et al. Small-file access in parallel file systems [C] //Processdings of the 23rd IEEE International Parallel and Distributed Processing Symposium, 2009. 被引量:1

引证文献3

二级引证文献8

相关作者

内容加载中请稍等...

相关机构

内容加载中请稍等...

相关主题

内容加载中请稍等...

浏览历史

内容加载中请稍等...
;
使用帮助 返回顶部