分布式存储系统作为数据存储的载体,广泛应用于大数据领域.纠删码存储方式相对副本方式,既具有较高的空间利用效率,又能保证数据存储的可靠性,因此被越来多的应用于存储系统当中.在EB级大规模纠删码分布式存储系统中,元数据管理成本较大...分布式存储系统作为数据存储的载体,广泛应用于大数据领域.纠删码存储方式相对副本方式,既具有较高的空间利用效率,又能保证数据存储的可靠性,因此被越来多的应用于存储系统当中.在EB级大规模纠删码分布式存储系统中,元数据管理成本较大,位置信息等元数据查询效率影响了I O时延和吞吐量.基于位置信息记录的有中心数据放置算法需要频繁访问元数据服务器,导致性能优化受限,基于Hash映射的无中心数据放置算法越来越多地得到应用.但面向纠删码的无中心放置算法,在节点变更和数据恢复过程中,存在位置变更困难、迁移数据量大、数据恢复和迁移并发度低等问题.提出了一种基于条带的一致性Hash数据放置算法(consistent Hash data placement algorithm based on stripe,SCHash),SCHash以条带为单位放置数据,通过把数据块到节点的映射转化为条带到节点组的映射过程,减少节点变动过程中的数据迁移量,从而在恢复过程中降低了变动数据的比例,加速了恢复带宽.并基于SCHash算法设计了一种基于条带的并发I O调度恢复策略,通过避免选取同一节点的数据块进行I O操作,提升了I O并行度,通过调度恢复I O和迁移I O的执行顺序,减少了数据恢复的执行时间.相比APHash数据放置算法,SCHash在数据恢复过程中,减少了46.71%~85.28%数据的迁移.在条带内重建时,恢复带宽提升了48.16%,在条带外节点重建时,恢复带宽提升了138.44%.展开更多
Current blockchain consensus protocols have a triangle of contradictions in aspects of decentralization, security, and energy consumption, and cannot be synchronously optimized. We describe a design of two new blockch...Current blockchain consensus protocols have a triangle of contradictions in aspects of decentralization, security, and energy consumption, and cannot be synchronously optimized. We describe a design of two new blockchain consensus protocols, called "CHB-consensus" and "CHBD-consensus," based on a consistent hash algorithm. Honest miners can fairly gain the op-portunity to create blocks. They do not consume any extra computational power resources when creating new blocks, and such blocks can obtain the whole blockchain network to confirm consensus with fairness. However, malicious miners have to pay massive computational power resources for attacking the new block creation privilege or double-spending. Blockchain networks formed by CHB-consensus and CHBD-consensus are based on the same security assumption as that in Bitcoin systems, so they save a huge amount of power without sacrificing decentralization or security. We analyze possible attacks and give a rigorous but adjustable validation strategy. CHB-consensus and CHBD-consensus introduce a certification authority (CA) system, which does not have special management or control rights over blockchain networks or data structures, but carries the risk of privacy breaches depending on credibility and reliability of the CA system. Here, we analyze the robustness and energy consumption of CHB-consensus and CHBD-consensus, and demonstrate their advantages through theoretical derivation.展开更多
为了解决Redis数据库复制性能差和扩展性差的问题,该文设计了一个Redis可扩展与高效复制方案,即基于文件共享的主从复制和基于一致性哈希的集群(RSCCH-Redis,Replication on Share and Clusters on Consistent Hash of Redis)方案.该方...为了解决Redis数据库复制性能差和扩展性差的问题,该文设计了一个Redis可扩展与高效复制方案,即基于文件共享的主从复制和基于一致性哈希的集群(RSCCH-Redis,Replication on Share and Clusters on Consistent Hash of Redis)方案.该方案在Redis应用中引入了一致性哈希以实现可扩展性及引入了基于文件共享的主从复制功能来提高复制性能;采用一致性哈希作为映射方式,并在其中引入了虚拟节点,这样不但保证了映射关系的单调性与分散性,还保证了映射的负载均衡性;采用文件共享的主从复制方式,可以避免初次复制时大量的数据传输,减轻网络压力,加快复制速度,从而减少故障恢复时间.实验数据表明,该文设计的Redis高可用方案RSCCH-Redis不但解决了Redis数据库复制性能差的问题,而且有着良好的可扩展性.展开更多
文摘分布式存储系统作为数据存储的载体,广泛应用于大数据领域.纠删码存储方式相对副本方式,既具有较高的空间利用效率,又能保证数据存储的可靠性,因此被越来多的应用于存储系统当中.在EB级大规模纠删码分布式存储系统中,元数据管理成本较大,位置信息等元数据查询效率影响了I O时延和吞吐量.基于位置信息记录的有中心数据放置算法需要频繁访问元数据服务器,导致性能优化受限,基于Hash映射的无中心数据放置算法越来越多地得到应用.但面向纠删码的无中心放置算法,在节点变更和数据恢复过程中,存在位置变更困难、迁移数据量大、数据恢复和迁移并发度低等问题.提出了一种基于条带的一致性Hash数据放置算法(consistent Hash data placement algorithm based on stripe,SCHash),SCHash以条带为单位放置数据,通过把数据块到节点的映射转化为条带到节点组的映射过程,减少节点变动过程中的数据迁移量,从而在恢复过程中降低了变动数据的比例,加速了恢复带宽.并基于SCHash算法设计了一种基于条带的并发I O调度恢复策略,通过避免选取同一节点的数据块进行I O操作,提升了I O并行度,通过调度恢复I O和迁移I O的执行顺序,减少了数据恢复的执行时间.相比APHash数据放置算法,SCHash在数据恢复过程中,减少了46.71%~85.28%数据的迁移.在条带内重建时,恢复带宽提升了48.16%,在条带外节点重建时,恢复带宽提升了138.44%.
基金Project supported by the National Natural Science Foundation of China(Nos.61672499,61772502,and 61972382)the Key Special Project of Beijing Municipal Science&Technology Commission,China(No.Z181100003218018)+2 种基金the Natural Science Foundation of Inner Mongolia,Open Foundation of State Key Laboratory of Networking and Switching Technology of China(No.SKLNST-2016-2-09)the SV-ICT Blockchain&DAPP Joint Lab of Chinathe ICT-SSC Blockchain Joint Lab of China。
文摘Current blockchain consensus protocols have a triangle of contradictions in aspects of decentralization, security, and energy consumption, and cannot be synchronously optimized. We describe a design of two new blockchain consensus protocols, called "CHB-consensus" and "CHBD-consensus," based on a consistent hash algorithm. Honest miners can fairly gain the op-portunity to create blocks. They do not consume any extra computational power resources when creating new blocks, and such blocks can obtain the whole blockchain network to confirm consensus with fairness. However, malicious miners have to pay massive computational power resources for attacking the new block creation privilege or double-spending. Blockchain networks formed by CHB-consensus and CHBD-consensus are based on the same security assumption as that in Bitcoin systems, so they save a huge amount of power without sacrificing decentralization or security. We analyze possible attacks and give a rigorous but adjustable validation strategy. CHB-consensus and CHBD-consensus introduce a certification authority (CA) system, which does not have special management or control rights over blockchain networks or data structures, but carries the risk of privacy breaches depending on credibility and reliability of the CA system. Here, we analyze the robustness and energy consumption of CHB-consensus and CHBD-consensus, and demonstrate their advantages through theoretical derivation.
文摘为了解决Redis数据库复制性能差和扩展性差的问题,该文设计了一个Redis可扩展与高效复制方案,即基于文件共享的主从复制和基于一致性哈希的集群(RSCCH-Redis,Replication on Share and Clusters on Consistent Hash of Redis)方案.该方案在Redis应用中引入了一致性哈希以实现可扩展性及引入了基于文件共享的主从复制功能来提高复制性能;采用一致性哈希作为映射方式,并在其中引入了虚拟节点,这样不但保证了映射关系的单调性与分散性,还保证了映射的负载均衡性;采用文件共享的主从复制方式,可以避免初次复制时大量的数据传输,减轻网络压力,加快复制速度,从而减少故障恢复时间.实验数据表明,该文设计的Redis高可用方案RSCCH-Redis不但解决了Redis数据库复制性能差的问题,而且有着良好的可扩展性.
