期刊文献+

NVMMDS——一种面向非易失存储器的元数据管理方法 被引量:3

NVMMDS—Metadata Management Method Based on Non-Volatile Memory
下载PDF
导出
摘要 元数据管理方法是影响文件系统性能的重要因素.针对现有元数据管理方法存在的查找性能低、适应性差和丢失元数据等问题,设计了面向非易失存储器的元数据管理方法(NVMMDS).首先针对元数据的访问特性和管理要求,给出了NVMMDS的结构和元数据管理流程,混合使用非易失存储器和DRAM存储元数据,为提高元数据查找性能和避免丢失元数据奠定了基础;设计了基于NVBB树的元数据查找算法和基于主动写回的元数据缓存算法,提高了元数据的查找性能,增强了元数据管理方法的适应能力,避免了元数据丢失问题.与现有元数据管理方法进行了分析和比较,在单机文件系统ReiserFs和分布式文件系统pNFS中实现了NVMMDS原型,使用FileBench和多个标准数据集进行了测试与分析,验证了NVMMDS能提高文件系统最大35%的操作处理速度和I/O性能. Metadata management methods are important factors to affect the performance of file system. A novel metadata management method based on non-volatile memory (NVMMDS) is designed to concern the efficiency and adaptability of current metadata management methods. The structure of NVMMDS and metadata management flow are presented according to the metadata accessing characteristic and management demands. The metadata is stored in non-volatile memory and DRAM is used to cache the update metadata, which provides the foundation for improving performance, avoiding metadata loss and enhancing the metadata operation atomicity. A metadata lookup algorithm based on NVBB-tree is used to uniformly search the metadata in non-volatile memory and metadata cache in DRAM with low overhead. A metadata cache algorithm based on active writing back is used to extend the life of non-volatile memory and avoid the loss of metadata. Compared with the current metadata management methods, our algorithms greatly improve the efficiency and adaptability of metadata management, and avoid the loss of metadata. The NVMMDS prototype is realized on ReiserFs and pNFS. The FileBench and several standard data sets are used to evaluate. The experimental results show that NVMMDS can improve maximum 35% IOPS and I/O performance of file system.
出处 《计算机研究与发展》 EI CSCD 北大核心 2013年第1期69-79,共11页 Journal of Computer Research and Development
基金 国家自然科学基金项目(61003288) 高等学校博士学科点专项科研基金项目(20093227110005) 广东省自然科学基金项目(S2011010006118) 江苏省高校自然科学基金项目(09KJB520001)
关键词 文件系统 元数据管理方法 非易失存储器 元数据查找方法 元数据缓存 file system metadata management method non-volatile memory metadata lookup method metadata cache
  • 相关文献

参考文献23

  • 1Satyanarayanan M, Kistler J J, Kumar P, et al. Coda: A highly available file system for distributed workstation environments[J]. IEEETransonComputers, 1990, 39(4): 447-459. 被引量:1
  • 2Li Weijia, Xue Wei, Shu Jiwu, et al. Dynamic hashing: Adaptive mctadata management for petabyte-scalc file systems [C] //Proc of IEEE MSST'2006. Los Alamitos, CA: IEEE Computer Society, 2006:93-98. 被引量:1
  • 3Zhu Yifeng, Jiang Hong, Wang Jun, et al. HBA: Distributed metadata management for large cluster-based storage systems [J]. IEEE Trans on Parallel and Distributed Systems, 2008, 19(6):750-763. 被引量:1
  • 4Riedel E, Kallahalla M, Swaminathan R. A framework for evaluating storage system security [C] //Proe of Usenix FAST'2002. Berkeley, CA- USENIX Association, 2002: 15 -30. 被引量:1
  • 5Weil S A, Pollack K T, Brandt S A, et aI. Dynamic metadata management for petabyte-scale file systems [C] // Proc of ACM/IEEE SC'04. Piscataway, NJ - IEEE Computer Society, 2004:523-534. 被引量:1
  • 6Leung A W, Shao M, Bisson T, et al. Spyglass: Fast, scalable metadata .search for large-scale storage systems [C] //Proc of the 7th Conf on File and Storage Technologies. Berkeley, CA: USENIX Association, 2009:153-166. 被引量:1
  • 7穆飞,薛巍,舒继武,郑纬民.基于定位目录的元数据管理方法[J].清华大学学报(自然科学版),2009(8):1229-1232. 被引量:6
  • 8Brandt S A, Miller E L, Long DaDE, et al. Efficient metadata management in large distributed file systems [C] // Proc of the 20th IEEE/llth NASA Goddard Conf on Mass Storage System and Technologies. Los Alamitos,CA: IEEE Computer Society, 2003:290-297. 被引量:1
  • 9Woodhouse D. JFFS= The journalling flash file system [R]. North Carolina: RedHat Inc, 2001. 被引量:1
  • 10Wang Yongkun, Goda K, Nakano M, et al. Early experience and evaluation of file systems on SSD with database applications [C] //Proc of the 5th IEEE Int Conf on Networking, Architecture, and Storage. Los Alamitos, CA: IEEE Computer Society, 2010:467-476. 被引量:1

二级参考文献19

  • 1Roselli D, Lorch J, Anderson T. A comparison of file system workloads [C]// Proceedings of the 2000 USENIX Annual Technical Conference. San Diego, California, USA, 2000:41 - 54. 被引量:1
  • 2Ji M, Felten E, Wang R, et al. An island-based file system for highly available and scalable Internet services [C]// Proceedings of the 4th USENIX Windows Systems Symposium. Seattle, Washington, USA, 2000: 37-48. 被引量:1
  • 3Memik G, Kandemir M, Choudhary A. Exploiting inter-file access patterns using multi-collective I/O [C]// Proc of USENIX Conference on File and Storage Technologies (FAST). Monterey, CA, USA, 2002: 245-258. 被引量:1
  • 4LI Weijia, XUE Wei, SHU Jiwu, et al. Dynamic hashing: Adaptive metadata management for Petabyte-scale file systems [C]// Proceedings of the 23th IEEE / 14th NASA Goddard Conference on Mass Storage Systems and Technologies. Washington, DC, USA: IEEE Computer Society, 2006:51 - 56. 被引量:1
  • 5Weil S, Pollack K, Brandt S, et al. Dynamic metadata management for Petabyte-scale file systems [C]// Proceedings of the 2004 ACM/IEEE Conference on Supercomputing (SC'04). Washington, DC, USA: IEEE Computer Society, 2004:47-54. 被引量:1
  • 6Well S, Brandt S, Miller E, et al. Ceph; A scalable, high-performance distributed file system [C]// Proceedings of the 7th Conference on Operating Systems Design and Implementation (OSDI'06). Seattle, Washington, USA, 2006; 307 - 320. 被引量:1
  • 7Roselli D, Lorch J, Anderson T. A comparison of file system workloads [C]// Proceedings of the 2000 USENIX Annual Technical Conference. 2000:41 -54. 被引量:1
  • 8Wang F, Xin Q, Hong B, et al. File system workload analysis for large scientific computing applications [C]//NASA/IEEE Conference on Mass Storage Systems and Technologies (MSST 2004). Adelphi, MD, USA, 2004: 139 - 152. 被引量:1
  • 9XtreemFS:An open source distributed and replicated file system for the cloud. http://www.xtreemfs.org/faq.php . 2012 被引量:1
  • 10Soundararajan G,Prabhakaran V,Balakrishan M,et al.Extending SSD lifetimes with disk-based Write caches. Proc of the8th USENIX Conf on File and Storage Technologies . 2010 被引量:1

共引文献11

同被引文献36

  • 1刘爱贵.分布式文件系统元数据服务模型. 被引量:1
  • 2Wikipedia. Google file system[EB/OL].http://en.wi-kipedia.org/wiki/Google_File_System,2013. 被引量:1
  • 3The Apache Software Foundation. HDFS architecture guide[EB/OL].http://hadoop.apache.org/common/docs/current/hdfs_design.html,2011. 被引量:1
  • 4Oracle. Lustre internals documentation[EB/OL].http://wiki.lustre.org/lid/index.html,2010. 被引量:1
  • 5刘爱贵.GlusterFS集群文件系统研究. 被引量:1
  • 6董西成.HDFS Federation 设计动机与基本原理. 被引量:1
  • 7Zaharia M,Chowdhury M,Das T. Resilient distributed datasets,a fault-tolerant abstraction for in-memory duster computing[R].San Francisco:University of California at Berkeley,Technical Report No.UCB/EECS-2011-82,2011. 被引量:1
  • 8Demehenko Y, Grosso P, de Laat C, et al. Addressing big data issues in scientific data infrastructure [C] //Proc of 2013 Int Conf on Collaboration Technologies and Systems. Piscataway, NJ: IEEE, 2013:48-55. 被引量:1
  • 9Zheng M, Tucek J, Qin F, et al. Understanding the robustness of SSDS under power fault [C] //Proc of the llth USENIX Conf on File and Storage Technologies. Berkeley, CA: USENIX Association, 2013; 271-284. 被引量:1
  • 10Witkowski T. Formal verification of Linux device drivers [D]. Dresden, Germany: Dresden University of Technology, 2007. 被引量:1

引证文献3

二级引证文献16

相关作者

内容加载中请稍等...

相关机构

内容加载中请稍等...

相关主题

内容加载中请稍等...

浏览历史

内容加载中请稍等...
;
使用帮助 返回顶部