期刊文献+

基于Hadoop框架下的Fast-Newman算法改进

Improvement of Fast-Newman Algorithm Based on Hadoop Framework
下载PDF
导出
摘要 Fast-Newman算法的复杂程度高,尤其是在计算模块度(Modularity)时,在边数较多的情况下,随着结点数提高,极大的影响着计算速度。为此,本文提出了一种基于Hadoop框架下的改进策略。该策略通过结点-边信息的划分,完成一定程度的分布化,在利用大量mappers的基础上,降低每次迭代时间,从而最终提升计算速度。通过对Zachary网络与随机ego-Facebook部分集的实验对比可以发现,算法加速比与并行序列数量有关。 To cut down the complexity of the fast-newman algorithm, especially the computation of 'modularity',which raises rapidly with the larger edges, a distributed fast-newman based on Hadoop framework has been proposed in this paper. It reduces the computing cost by degrading the number of pairs of edge and nodes to realize the computing parallel with matched count of mappers(computers). By recording the experiments of Zachary-net and the part of ego-Facebook, the relationship of speed-up ratio and numbers of mappers has been found.
出处 《科技广场》 2016年第11期9-12,共4页 Science Mosaic
关键词 HADOOP Fast-newman 分布式 社区发现 Hadoop Fast-newman Distributed Community Discovery
  • 相关文献

参考文献8

二级参考文献100

  • 1Ghemawat S, Gobioff H, Leung ST. The Google file system. In: Proc. of the SOSP 2003. 2003.20-43. [doi: 10.1145/1165389. 945450]. 被引量:1
  • 2Dean J, Ghemawat S. MapReduce: Simplified data processing on large clusters. In: Proc. of the OSDI 2004. 2004. 137-150. [doi: 10.1145/1327452.1327492]. 被引量:1
  • 3Yang HC, Dasdan A, Hsiao RL, Parker DS. Map-Reduce-Merge: Simplified relational data processing on large cluster. In: Proc. of the SIGMOD 2007. 2007. 1029-1040. [doi: 10.1145/1247480.1247602]. 被引量:1
  • 4Lammel R. Google's MapReduce programming model Revisited. Science Computer Program, 2008,70(1):1-30. [doi: 10.1016/ j .scico .2007.07.001 ]. 被引量:1
  • 5Thusoo A, Sarma JS, Jain N, Shao Z, Chakka P, Anthony S, Liu H, Wyckoff P, Murthy R. Hi:ce: A warehousing solution over a map-reduce framework. Proc. of the VLDB Endowment, 2009,2(2): 1626-1627. 被引量:1
  • 6Thusoo A, Sarma JS, Jain N, Shao Z, Chakka P, Zhang N, Antony S, Liu H, Murthy R. Hive--A petabyte scale data warehouse using Hadoop data engineering. In: Proc. of the ICDE. 2010. 996-1005. [doi: 10.1109/ICDE.2010.5447738]. 被引量:1
  • 7Olston C, Reed B, Sirvastava U, Kumar R, Tomkins A. Pig Latin: A not-so-foreign language for data processing. In: Proc. of the SIGMOD. 2008. 1099-1110. [doi: 10.1145/1376616.1376726]. 被引量:1
  • 8White T. Hadoop: The Definitive Guide. O'Reilly, 2009. 被引量:1
  • 9Apache Hadoop. http://hadoop.apache.org/. 被引量:1
  • 10Murty J. Programming Amazon Web Services: S3, EC2, SQS, FPS, and SimpleDB. O'Reilly, 2008. 被引量:1

共引文献399

相关作者

内容加载中请稍等...

相关机构

内容加载中请稍等...

相关主题

内容加载中请稍等...

浏览历史

内容加载中请稍等...
;
使用帮助 返回顶部