由于广域测量系统(wide area measurement system,WAMS)海量实时数据的大规模、高负荷并发访问,其采用的同步机制在很大程度上约束了系统的效率。针对于目前普遍使用的整体锁定机制带来的由于访问串行化导致的效率低下问题,提出了粒度...由于广域测量系统(wide area measurement system,WAMS)海量实时数据的大规模、高负荷并发访问,其采用的同步机制在很大程度上约束了系统的效率。针对于目前普遍使用的整体锁定机制带来的由于访问串行化导致的效率低下问题,提出了粒度可控并发访问同步算法(controllablegranularity concurrency synchronization,CGCS),该方法使用控制标志位建立基于子集超集依赖的锁定条件和先进先出等待队列,并可控锁定级别,实现对实时数据访问的互斥粒度粗细的任意控制,同时作用于整体文件、表、元组,进而任务可以最大化地并发执行。通过实验,给出了系统的并发处理能力和IO响应能力的测试过程和结果,证明了CGCS算法在并发高、访问散的情况下能充分发挥CPU的并行处理能力,使WAMS系统的效率得到大幅提升。展开更多
Building a cloud geodatabase for a sponge city is crucial to integrate the geospatial information dispersed in various departments for multi-user high concurrent access and retrieval,high scalability and availability,...Building a cloud geodatabase for a sponge city is crucial to integrate the geospatial information dispersed in various departments for multi-user high concurrent access and retrieval,high scalability and availability,efficient storage and management.In this study,Hadoop distributed computing framework,including Hadoop distributed file system and MapReduce(mapper and reducer),is firstly designed with a parallel computing framework to process massive spatial data.Then,access control with a series of standard application programming interfaces for different functions is designed,including spatial data storage layer,cloud geodatabase access layer,spatial data access layer and spatial data analysis layer.Subsequently,a retrieval model is designed,including direct addressing via file name,three-level concurrent retrieval and block data retrieval strategies.Main functions are realised,including real-time concurrent access,high-performance computing,communication,massive data storage,efficient retrieval and scheduling decisions on the multi-scale,multi-source and massive spatial data.Finally,the performance of Hadoop cloud geodatabases is validated and compared with that of the Oracle database.The cloud geodatabase for the sponge city can avoid redundant configuration of personnel,hardware and software,support the data transfer,model debugging and application development,and provide accurate,real-time,virtual,intelligent,reliable,elastically scalable,dynamic and on-demand cloud services of the basic and thematic geographic information for the construction and management of the sponge city.展开更多
基金Project(NZ1628)supported by the Natural Science Foundation of Ningxia,China
文摘Building a cloud geodatabase for a sponge city is crucial to integrate the geospatial information dispersed in various departments for multi-user high concurrent access and retrieval,high scalability and availability,efficient storage and management.In this study,Hadoop distributed computing framework,including Hadoop distributed file system and MapReduce(mapper and reducer),is firstly designed with a parallel computing framework to process massive spatial data.Then,access control with a series of standard application programming interfaces for different functions is designed,including spatial data storage layer,cloud geodatabase access layer,spatial data access layer and spatial data analysis layer.Subsequently,a retrieval model is designed,including direct addressing via file name,three-level concurrent retrieval and block data retrieval strategies.Main functions are realised,including real-time concurrent access,high-performance computing,communication,massive data storage,efficient retrieval and scheduling decisions on the multi-scale,multi-source and massive spatial data.Finally,the performance of Hadoop cloud geodatabases is validated and compared with that of the Oracle database.The cloud geodatabase for the sponge city can avoid redundant configuration of personnel,hardware and software,support the data transfer,model debugging and application development,and provide accurate,real-time,virtual,intelligent,reliable,elastically scalable,dynamic and on-demand cloud services of the basic and thematic geographic information for the construction and management of the sponge city.
