摘要
数据探索作为数据分析的一个重要环节,必须能够高效的获取数据集的关键性指标,比如最大/最小值、均值等.关系数据库中这些指标可以通过SQL语句的聚集函数得到.为了实现海量数据下的高效聚集,关系数据库领域学者提出了在线聚集.在大数据时代,云环境下的在线聚集技术开始得到重视.但是目前云环境下的在线聚集研究基本是针对Count、Sum等聚集函数,尚未有针对Max/Min在线聚集的研究.本文利用切比雪夫不等式和中心极限定理,通过分位数来衡量Max/Min在线聚集的精确度.实验证明,该方法能够很好的适应大数据环境下的在线聚集,并具有良好的扩展性.
As an important part of data analysis, data exploration must be able to efficiently access key indicators of data sets, such as max/min, average and etc. These indicators can be obtained by SQL aggregate functions in relational database. In order to achieve this goal in massive dataset, scholars have proposed the concept of onlineaggregation. In the era of big data, online aggregation in the cloud has attracted attentions. Most of the research focuses on the aggregation function such as Count, Sum and other aggregate functions, while there is little works on the Max/Min online aggregation now. In this paper, we use quantile to measure the accuracy of Max/Min online aggregation which induced by chebyshev＇s inequality and central limit theorem. The experimental results demonstrate the efficiency of the method and it can well adapt to online aggregation for big data.
出处
《小型微型计算机系统》
CSCD
北大核心
2015年第10期2177-2182,共6页
Journal of Chinese Computer Systems
基金
国家自然科学基金项目(61379050
91224008)资助
国家"八六三"高技术研究发展计划项目(2013AA013204)资助
高等学校博士学科点专项科研基金课题项目(20130004130001)资助
中国人民大学科学研究基金项目(11XNL010)资助
关键词
在线聚集
云计算
切比雪夫不等式
中心极限定理
online aggregation
cloud computing
chebyshev＇ s inequality
central limit theorem
