摘要
地面自动站气象数据的量级、观测传输频次日益增长,传输方式从传统的文本方式发展到消息传输,对数据处理时效有了更高的要求。该文提出采用Storm流式技术进行地面自动站气象数据的解码和入库,使用Spout节点连接外部数据源,将数据转化为Tuple,传递给解码Bolt、入库Bolt和DI/EI消息Bolt,分别进行实时的入库和监控,且达到灵活调整系统资源的目的。该应用与传统的简约流程对比,稳定性、处理效率、拓展性都有了显著提高。
The quantity and observational frequency of meteorological data by automatic weather station are increasing. The transmission mode has developed from the traditional text mode to the message transmission mode. These changes make higher requirement for data processing timeliness. In this paper,we used Storm Flow Technology on decoding and warehouse of automatic weather station data. It used Spout to connect external data sources,and convert data into Tuple,and then transmit to Bolt-decode,Bolt-warehouse and Bolt-DIEI. They can be used to real-time warehouse,monitoring,and quickly adjust system resource. Compared with the traditional simplified process,the stability,processing efficiency and expansibility of the application have been greatly improved.
作者
廖婷婷
王彪
肖卫青
李从英
郭茜
LIAO Tingting;WANG Biao;XIAO Weiqing;LI Congying;GUO Qian(Guizhou Meteorological Information Center,Guiyang 550002,China;National Meteorological Information Center,Beijing 100081,China)
出处
《中低纬山地气象》
2019年第5期78-81,共4页
Mid-low Latitude Mountain Meteorology
基金
贵州省科技计划项目([2019]2386):基于大数据的气象防灾减灾服务平台研发
关键词
地面气象数据
海量数据
Storm流式技术
高效处理
automatic weather station data
big data
storm flow technology
high-activity processing
