摘要
对象存储并行文件系统在高性能计算领域被广泛采用,虽然大文件访问性能优异,但其小文件访问存在局限性。通过分析典型对象存储并行文件系统下的小文件的访问开销,采用服务端小文件专用缓存和基于对象命名关联性预取的方法提高小文件的缓存命中率,降低对象存储架构下小文件的读取延迟。基于Lustre并行文件系统实现了该方法。实际系统测试表明,该方法可显著提高小文件访问性能,其中小文件打包性能提升最大可达88%。
Object based parallel file systems are widely used in current supercomputers for its excellent performance of large file at the expense of its lower efficiency on small file access.Through analysis of the overhead of small file access on typical object based parallel file system,a server-side small file aware cache is proposed in order to reduce file access latency by aggressively prefetching a set of related files object and improving small file hit rate.This method is implemented and evaluated in Lustrefile system which is a popular parallel file system for HPC system.The experiments in real system show that it can improve the performance of small file access dramatically,especially with the reduction of up to 88％ of total cost in scenario of file archiving operation.
出处
《计算机工程与科学》
CSCD
北大核心
2013年第12期8-13,共6页
Computer Engineering & Science
基金
国家自然科学基金资助项目(61120106005)
国家863计划资助项目(2012AA01A301)
