摘要
随着船舶数量不断增加,AIS数据的数量持续增长,与此同时数据复杂和处理难度大等缺陷也日渐突出。为了分析大量的AIS数据并充分挖掘有效信息,文中构建了一个基于Spark、Hadoop和Mesos的AIS数据处理和分析平台,使用k-means算法对船舶数据进行聚类分析,获得更清晰的船舶航迹信息,并比较有、无安装Spark程序的计算机进行聚类分析时所需的时间。从实验结果看出,与非大数据平台相比,Spark集群中AIS船舶数据点的聚类分析速度大大提高。
With the increasing number of ships,the number of AIS data continues to grow.At the same time,the data complexity and processing difficulty are also increasingly prominent.In order to analyze a large number of AIS data and fully mine the effective information,an AIS data processing and analysis platform is constructed based on Spark,Hadoop and Mesos,K-means algorithmis used to cluster the ship data and obtain clearer ship track information.In addition,the time required for cluster analysis with or without Spark is compared.The experimental results indicate that the speed of cluster analysis of AIS ship data points in Spark cluster is greatly improved by comparison with the non-big data platform.
作者
初秀民
林宏
王志远
CHU Xiumin;LIN Hong;WANG Zhiyuan(College of Physics and Electronic Information Engineering, Minjiang University, Fuzhou 350108, China;Fujian Engineering Research Center of Safety Control for Ship Smart Navigation, Minjiang University, Fuzhou 350108, China;Fujian College’s Engineering Research Center of Marine Smart Ship Equipment, Minjiang University, Fuzhou 350108, China)
出处
《交通科技》
2021年第3期138-141,共4页
Transportation Science & Technology
