固态盘(solid state drive,SSD)因为其优越的性能已被大量部署于当前的存储系统中.但是,由于寿命有限,SSD的可靠性受到广泛的质疑.磁盘阵列(redundant arrays of inexpensive disk,RAID)是一种传统的用来提高可靠性的手段,但并不适用于S...固态盘(solid state drive,SSD)因为其优越的性能已被大量部署于当前的存储系统中.但是,由于寿命有限,SSD的可靠性受到广泛的质疑.磁盘阵列(redundant arrays of inexpensive disk,RAID)是一种传统的用来提高可靠性的手段,但并不适用于SSD.这项工作提出一种基于SSD和磁盘的混合存储系统,构建该系统的主要思想是SSD响应所有I/O请求,从而获得较高的性能;磁盘备份所有数据,从而保证系统的可靠性.但是,磁盘的I/O性能显著低于SSD,构建该系统的问题在于磁盘能否及时地备份SSD上的数据.为了解决这一问题,从两方面提出优化:在延迟方面,采用非易失主存弥补磁盘与SSD的延迟差距;在带宽方面,采用两种措施:1)在单块磁盘内部重组I/O请求,使磁盘尽可能的顺序读写;2)采用多块磁盘备份多块SSD,通过将一块SSD上的写请求分散到多块磁盘上,有效应对单块SSD上出现的突发写请求.通过原型系统实现表明,该混合系统是可行的:磁盘能够为SSD提供实时的数据备份;与其他系统相比,该混合系统取得较高的性价比.展开更多
以SSD(solid state drive)为代表的新型存储介质在虚拟化环境下得到了广泛的应用,通常作为虚拟机读写缓存,起到优化磁盘I/O性能的作用.已有研究往往关注SSD缓存的容量规划,依据缓存读写命中率评价SSD缓存分配效果,未能充分考虑SSD的服...以SSD(solid state drive)为代表的新型存储介质在虚拟化环境下得到了广泛的应用,通常作为虚拟机读写缓存,起到优化磁盘I/O性能的作用.已有研究往往关注SSD缓存的容量规划,依据缓存读写命中率评价SSD缓存分配效果,未能充分考虑SSD的服务能力上限,难以适用于典型的分布式应用场景,存在虚拟机抢占SSD缓存资源,导致虚拟机中应用性能违约的可能.实现了虚拟化环境下面向多目标优化的自适应SSD缓存系统,考虑了SSD的服务能力上限.基于自适应闭环实现对虚拟机和应用状态的动态感知.动态检测局部SSD缓存抢占状态,基于聚类方法生成虚拟机的优化放置方案,依据全局SSD缓存供给能力确定虚拟机迁移顺序和时机.实验结果表明,该方法在应对典型分布式应用场景时可以有效缓解SSD缓存资源的争用,同时满足应用对虚拟机放置的需求,提升应用的性能并兼顾应用的可靠性.在Hadoop应用场景下,平均降低了25%的任务执行时间,对I/O密集型应用平均提升39%的吞吐率.在Zoo Keeper应用场景下,以不到5%的性能损失为代价,应对了虚拟化主机的单点失效带来的虚拟机宕机问题.展开更多
基于固态硬盘(solid-state drive,SSD)和硬盘(hard disk drive,HDD)混合存储的数据中心已经成为大数据计算领域的高性能载体,数据中心负载应该可将不同特性的数据按需持久化到SSD或HDD,以提升系统整体性能.Spark是目前产业界广泛使用的...基于固态硬盘(solid-state drive,SSD)和硬盘(hard disk drive,HDD)混合存储的数据中心已经成为大数据计算领域的高性能载体,数据中心负载应该可将不同特性的数据按需持久化到SSD或HDD,以提升系统整体性能.Spark是目前产业界广泛使用的高效大数据计算框架,尤其适用于多次迭代计算的应用领域,其原因在于Spark可以将中间数据持久化在内存或硬盘中,且持久化数据到硬盘打破了内存容量不足对数据集规模的限制.然而,当前的Spark实现并未专门提供显式的面向SSD的持久化接口,尽管可根据配置信息将数据按比例分布到不同的存储介质中,但是用户无法根据数据特征按需指定RDD的持久化存储介质,针对性和灵活性不足.这不仅成为进一步提升Spark性能的瓶颈,而且严重影响了混合存储系统性能的发挥.有鉴于此,首次提出面向SSD的数据持久化策略.探索了Spark数据持久化原理,基于混合存储系统优化了Spark的持久化架构,最终通过提供特定的持久化API实现用户可显式、灵活指定RDD的持久化介质.基于SparkBench的实验结果表明,经本方案优化后的Spark与原生版本相比,其性能平均提升14.02%.展开更多
SSD(solid state drive)的写入寿命比较有限,因此除命中率外,SSD缓存设备的写入量成为评价缓存替换算法的另一个关键指标。如何使算法提高写入数据转化为缓存命中的效率,从而延长SSD的使用寿命,具有重要的研究意义。目前,已有缓存替换...SSD(solid state drive)的写入寿命比较有限,因此除命中率外,SSD缓存设备的写入量成为评价缓存替换算法的另一个关键指标。如何使算法提高写入数据转化为缓存命中的效率,从而延长SSD的使用寿命,具有重要的研究意义。目前,已有缓存替换算法的设计一般基于时间局部性,即刚被访问的数据短期内被访问的概率较高,因此需要频繁的数据更新和较高写入量来保证较高命中率;或是通过不低的开销屏蔽相对最差的部分数据来减少一定的写入量,还缺少用低开销获得数据长期热度规律,有效提高缓存数据质量的算法。提出了访问序列折叠的缓存替换算法,用比较低的开销定位拥有长期稳定热度的数据写入缓存,明显提高了SSD缓存数据质量,在保证命中率的同时减少了SSD的写入量。实验表明,访问序列折叠算法相比LRU(least recently used)算法可在命中率损失低于10%的情况下减少90%的写入量,与SieveStore、L2ARC(level2 adjustable replacement cache)等写入优化缓存算法相比,命中率相当时可将写入量减少50%以上,有效达到了通过缓存高质量数据,减少SSD的写入量,延长其使用寿命的目的。展开更多
Emerging non-volatile memory technologies,especially flash-based solid state drives(SSDs),have increasingly been adopted in the storage stack.They provide numerous advantages over traditional mechanically rotating har...Emerging non-volatile memory technologies,especially flash-based solid state drives(SSDs),have increasingly been adopted in the storage stack.They provide numerous advantages over traditional mechanically rotating hard disk drives(HDDs)and have a tendency to replace HDDs.Due to the long existence of HDDs as primary building blocks for storage systems,however,much of the system software has been specially designed for HDD and may not be optimal for non-volatile memory media.Therefore,in order to realistically leverage its superior raw performance to the maximum,the existing upper layer software has to be re-evaluated or re-designed.To this end,in this paper,we propose PASS,an optimized I/O scheduler at the Linux block layer to accommodate the changing trend of underlying storage devices toward flash-based SSDs.PASS takes the rich internal parallelism in SSDs into account when dispatching requests to the device driver in order to achieve high performance.Specifically,it partitions the logical storage space into fixed-size regions(preferably the component package sizes)as scheduling units.These scheduling units are serviced in a round-robin manner and for every chance that the chosen dispatching unit issues only a batch of either read or write requests to suppress the excessive mutual interference.Additionally,the requests are sorted according to their visiting addresses while waiting in the dispatching queues to exploit high sequential performance of SSD.The experimental results with a variety of workloads have shown that PASS outperforms the four Linux off-the-shelf I/O schedulers by a degree of 3%up to41%,while at the same time it improves the lifetime significantly,due to reducing the internal write amplification.展开更多
Data layout in a file system is the organization of data stored in external storages. The data layout has a huge impact on performance of storage systems. We survey three main kinds of data layout in traditional file ...Data layout in a file system is the organization of data stored in external storages. The data layout has a huge impact on performance of storage systems. We survey three main kinds of data layout in traditional file systems： in-place update file system, log-structured file system, and copy-on-write file sys- tem. Each file system has its own strengths and weaknesses under different circumstances. We also include a recent us- age of persistent layout in a file system that combines both flash memory and byte- addressable non- volatile memory. With this survey, we conclude that persistent data layout in file systems may evolve dramatically in the era of emerging non-volatile memory.展开更多
Deduplication has been commonly used in both enterprise storage systems and cloud storage. To overcome the performance challenge for the selective restore operations of deduplication systems, solid-state-drive-based ...Deduplication has been commonly used in both enterprise storage systems and cloud storage. To overcome the performance challenge for the selective restore operations of deduplication systems, solid-state-drive-based （i.e., SSD-based） re^d cache cm, be deployed for speeding up by caching popular restore contents dynamically. Unfortunately, frequent data updates induced by classical cache schemes （e.g., LRU and LFU） significantly shorten SSDs＇ lifetime while slowing down I/O processes in SSDs. To address this problem, we propose a new solution -- LOP-Cache to greatly improve tile write durability of SSDs as well as I/O performance by enlarging the proportion of long-term popular （LOP） data among data written into SSD-based cache. LOP-Cache keeps LOP data in the SSD cache for a long time period to decrease the number of cache replacements. Furthermore, it prevents unpopular or unnecessary data in deduplication containers from being written into the SSD cache. We implemented LOP-Cache in a prototype deduplication system to evaluate its pertbrmance. Our experimental results indicate that LOP-Cache shortens the latency of selective restore by an average of 37.3% at the cost of a small SSD-based cache with only 5.56% capacity of the deduplicated data. Importantly, LOP-Cache improves SSDs＇ lifetime by a factor of 9.77. The evidence shows that LOP-Cache offers a cost-efficient SSD-based read cache solution to boost performance of selective restore for deduplication systems.展开更多
传统磁盘存储设备因其固有的机械特性,已不能满足当前的数据密集型应用程序的需求。基于闪存的固态存储设备(solid state drive,SSD)的出现改善了这种情况,并被广泛用作缓存以降低内存与磁盘之间的性能差距。针对由DRAM和SSD构成的多级...传统磁盘存储设备因其固有的机械特性,已不能满足当前的数据密集型应用程序的需求。基于闪存的固态存储设备(solid state drive,SSD)的出现改善了这种情况,并被广泛用作缓存以降低内存与磁盘之间的性能差距。针对由DRAM和SSD构成的多级缓存,提出了一种可配置的历史信息感知的多级缓存替换策略Charm.Charm允许用户配置应用的访问模式、读写模式等多项内容,并且还可以根据应用对文件的历史访问信息来判断访问模式,从而能够适应访问模式的变化.此外,Charm过滤掉那些只访问一次的数据,将多次访问的热数据缓存至SSD,减少对SSD的写入次数,提升SSD寿命.使用MCsim对Charm与现有的多级缓存替换算法进行了对比测试,在实际的工作负载下,Charm优于其它多级缓存算法.展开更多
文摘固态盘(solid state drive,SSD)因为其优越的性能已被大量部署于当前的存储系统中.但是,由于寿命有限,SSD的可靠性受到广泛的质疑.磁盘阵列(redundant arrays of inexpensive disk,RAID)是一种传统的用来提高可靠性的手段,但并不适用于SSD.这项工作提出一种基于SSD和磁盘的混合存储系统,构建该系统的主要思想是SSD响应所有I/O请求,从而获得较高的性能;磁盘备份所有数据,从而保证系统的可靠性.但是,磁盘的I/O性能显著低于SSD,构建该系统的问题在于磁盘能否及时地备份SSD上的数据.为了解决这一问题,从两方面提出优化:在延迟方面,采用非易失主存弥补磁盘与SSD的延迟差距;在带宽方面,采用两种措施:1)在单块磁盘内部重组I/O请求,使磁盘尽可能的顺序读写;2)采用多块磁盘备份多块SSD,通过将一块SSD上的写请求分散到多块磁盘上,有效应对单块SSD上出现的突发写请求.通过原型系统实现表明,该混合系统是可行的:磁盘能够为SSD提供实时的数据备份;与其他系统相比,该混合系统取得较高的性价比.
文摘以SSD(solid state drive)为代表的新型存储介质在虚拟化环境下得到了广泛的应用,通常作为虚拟机读写缓存,起到优化磁盘I/O性能的作用.已有研究往往关注SSD缓存的容量规划,依据缓存读写命中率评价SSD缓存分配效果,未能充分考虑SSD的服务能力上限,难以适用于典型的分布式应用场景,存在虚拟机抢占SSD缓存资源,导致虚拟机中应用性能违约的可能.实现了虚拟化环境下面向多目标优化的自适应SSD缓存系统,考虑了SSD的服务能力上限.基于自适应闭环实现对虚拟机和应用状态的动态感知.动态检测局部SSD缓存抢占状态,基于聚类方法生成虚拟机的优化放置方案,依据全局SSD缓存供给能力确定虚拟机迁移顺序和时机.实验结果表明,该方法在应对典型分布式应用场景时可以有效缓解SSD缓存资源的争用,同时满足应用对虚拟机放置的需求,提升应用的性能并兼顾应用的可靠性.在Hadoop应用场景下,平均降低了25%的任务执行时间,对I/O密集型应用平均提升39%的吞吐率.在Zoo Keeper应用场景下,以不到5%的性能损失为代价,应对了虚拟化主机的单点失效带来的虚拟机宕机问题.
文摘基于固态硬盘(solid-state drive,SSD)和硬盘(hard disk drive,HDD)混合存储的数据中心已经成为大数据计算领域的高性能载体,数据中心负载应该可将不同特性的数据按需持久化到SSD或HDD,以提升系统整体性能.Spark是目前产业界广泛使用的高效大数据计算框架,尤其适用于多次迭代计算的应用领域,其原因在于Spark可以将中间数据持久化在内存或硬盘中,且持久化数据到硬盘打破了内存容量不足对数据集规模的限制.然而,当前的Spark实现并未专门提供显式的面向SSD的持久化接口,尽管可根据配置信息将数据按比例分布到不同的存储介质中,但是用户无法根据数据特征按需指定RDD的持久化存储介质,针对性和灵活性不足.这不仅成为进一步提升Spark性能的瓶颈,而且严重影响了混合存储系统性能的发挥.有鉴于此,首次提出面向SSD的数据持久化策略.探索了Spark数据持久化原理,基于混合存储系统优化了Spark的持久化架构,最终通过提供特定的持久化API实现用户可显式、灵活指定RDD的持久化介质.基于SparkBench的实验结果表明,经本方案优化后的Spark与原生版本相比,其性能平均提升14.02%.
基金supported by the National Basic Research Program(973)of China(No.2004CB318203) the National High-Tech R&D Program(863)of China(No.2009AA01A402)+1 种基金the Natural Science Foundation of Hubei Province,China(No.2013CFB035)the Key Science Research Project of Hubei Education Office in China(No.D20141301)
文摘Emerging non-volatile memory technologies,especially flash-based solid state drives(SSDs),have increasingly been adopted in the storage stack.They provide numerous advantages over traditional mechanically rotating hard disk drives(HDDs)and have a tendency to replace HDDs.Due to the long existence of HDDs as primary building blocks for storage systems,however,much of the system software has been specially designed for HDD and may not be optimal for non-volatile memory media.Therefore,in order to realistically leverage its superior raw performance to the maximum,the existing upper layer software has to be re-evaluated or re-designed.To this end,in this paper,we propose PASS,an optimized I/O scheduler at the Linux block layer to accommodate the changing trend of underlying storage devices toward flash-based SSDs.PASS takes the rich internal parallelism in SSDs into account when dispatching requests to the device driver in order to achieve high performance.Specifically,it partitions the logical storage space into fixed-size regions(preferably the component package sizes)as scheduling units.These scheduling units are serviced in a round-robin manner and for every chance that the chosen dispatching unit issues only a batch of either read or write requests to suppress the excessive mutual interference.Additionally,the requests are sorted according to their visiting addresses while waiting in the dispatching queues to exploit high sequential performance of SSD.The experimental results with a variety of workloads have shown that PASS outperforms the four Linux off-the-shelf I/O schedulers by a degree of 3%up to41%,while at the same time it improves the lifetime significantly,due to reducing the internal write amplification.
基金supported by ZTE Industry-Academia-Research Cooperation Funds
文摘Data layout in a file system is the organization of data stored in external storages. The data layout has a huge impact on performance of storage systems. We survey three main kinds of data layout in traditional file systems： in-place update file system, log-structured file system, and copy-on-write file sys- tem. Each file system has its own strengths and weaknesses under different circumstances. We also include a recent us- age of persistent layout in a file system that combines both flash memory and byte- addressable non- volatile memory. With this survey, we conclude that persistent data layout in file systems may evolve dramatically in the era of emerging non-volatile memory.
基金This work is supported by the Natural Science Foundation of Beijing under Grant No. 4172031, the Pundamental Research FSmds for the Central Universities of China, and the Research Funds of Renmin University of China under Grant No. 16XNLQ02. Xiao Qin＇s work is supported by the U.S. National Science Foundation under Grant Nos. IIS-1618669, CCF-0845257 （CAREER）, CNS-0917137, CNS-0757778, CCF-0742187, CNS-0831502, CNS-0855251, and OCI-0753305. Xiao Qin＇s study is also supported by the Programme of Introducing Talents of Discipline to Universities （111 Project） in China under Grant No. B07038.
文摘Deduplication has been commonly used in both enterprise storage systems and cloud storage. To overcome the performance challenge for the selective restore operations of deduplication systems, solid-state-drive-based （i.e., SSD-based） re^d cache cm, be deployed for speeding up by caching popular restore contents dynamically. Unfortunately, frequent data updates induced by classical cache schemes （e.g., LRU and LFU） significantly shorten SSDs＇ lifetime while slowing down I/O processes in SSDs. To address this problem, we propose a new solution -- LOP-Cache to greatly improve tile write durability of SSDs as well as I/O performance by enlarging the proportion of long-term popular （LOP） data among data written into SSD-based cache. LOP-Cache keeps LOP data in the SSD cache for a long time period to decrease the number of cache replacements. Furthermore, it prevents unpopular or unnecessary data in deduplication containers from being written into the SSD cache. We implemented LOP-Cache in a prototype deduplication system to evaluate its pertbrmance. Our experimental results indicate that LOP-Cache shortens the latency of selective restore by an average of 37.3% at the cost of a small SSD-based cache with only 5.56% capacity of the deduplicated data. Importantly, LOP-Cache improves SSDs＇ lifetime by a factor of 9.77. The evidence shows that LOP-Cache offers a cost-efficient SSD-based read cache solution to boost performance of selective restore for deduplication systems.
文摘传统磁盘存储设备因其固有的机械特性,已不能满足当前的数据密集型应用程序的需求。基于闪存的固态存储设备(solid state drive,SSD)的出现改善了这种情况,并被广泛用作缓存以降低内存与磁盘之间的性能差距。针对由DRAM和SSD构成的多级缓存,提出了一种可配置的历史信息感知的多级缓存替换策略Charm.Charm允许用户配置应用的访问模式、读写模式等多项内容,并且还可以根据应用对文件的历史访问信息来判断访问模式,从而能够适应访问模式的变化.此外,Charm过滤掉那些只访问一次的数据,将多次访问的热数据缓存至SSD,减少对SSD的写入次数,提升SSD寿命.使用MCsim对Charm与现有的多级缓存替换算法进行了对比测试,在实际的工作负载下,Charm优于其它多级缓存算法.
