摘要
互联网数据的增长,催生了一大批新的数据处理技术,Map Reduce,Hadoop及相关技术使得我们能够处理的数据量比以前要大得多,但这些技术的设计目的都不是为了实时计算。然而随着社交网络服务的流行,大规模的实时数据处理已经越来越成为一种业务需求。Twitter Storm的出现弥补了Hadoop在实时处理方面的不足。本文就Storm的组成、运行机制以及计算模型进行研究,并设计与实现了基于Storm的社交网络中热门话题的实时计算问题。
With the growth of Internet data, a large number of new data processing techniques born. MapReduce, Hadoop and other related technologies enable us to handle much more data. But none of these technologies are designed to real-time computing. However, with the popularity of social networking service, real-time big data processing has increasing become a business needs. Twitter Storm’s appearance makes up for the lack of Hadoop in real-time processing. In this paper, we will study Storm composition, operation mechanism and computational models. What is more, we will design and implement the issue of real-time computing of the hottest topic in social network based on Storm.
出处
《软件》
2014年第10期16-20,共5页
Software
基金
国家科技支撑计划课题(2013BAH10F01)项目"劳动者全生命周期的就业信息服务系统及应用示范"
北京高等学校青年英才计划项目(YETP0445)
教育部信息网络工程研究中心和北京市教育委员会共建项目专项资助
