期刊文献+

大数据下张量充分降维方法及其应用研究 被引量:4

Research on Tensor Sufficient Dimension Reduction Method and Its Application in Big Data
下载PDF
导出
摘要 在大数据时代,金融学、基因组学和图像处理等领域产生了大量的张量数据。Zhong等(2015)提出了张量充分降维方法,并给出了处理二阶张量的序列迭代算法。鉴于高阶张量在实际生活中的广泛应用,本文将Zhong等(2015)的算法推广到高阶,以三阶张量为例,提出了两种不同的算法:结构转换算法和结构保持算法。两种算法都能够在不同程度上保持张量原有结构信息,同时有效降低变量维度和计算复杂度,避免协方差矩阵奇异的问题。将两种算法应用于人像彩图的分类识别,以二维和三维点图等形式直观展现了算法分类结果。将本文的结构保持算法与K-means聚类方法、t-SNE非线性降维方法、多维主成分分析、多维判别分析和张量切片逆回归共五种方法进行对比,结果表明本文所提方法在分类精度方面有明显优势,因此在图像识别及相关应用领域具有广阔的发展前景。 In the era of big data,a large amount of tensor data are generated in many fields such as finance,genomics and image processing.Zhong et al.(2015)propose tensor sufficient dimension reduction and present a sequential iterative algorithm for second-order tensors.In view of the wide application of high-order tensors in real life,this paper extends the iterative algorithm of Zhong et al.(2015)to higher-order tensors.Taking third-order tensors as an example,two different algorithms are proposed:structural transformation algorithm and structural maintenance algorithm.The two algorithms can effectively reduce the variable dimension and computational complexity while maintaining the original structural information of tensors,avoiding the singular problem of covariance matrix.The two algorithms are applied to the classification and recognition of color facial images.Then 2-D and 3-D scatter plots are used to display the classification results of the algorithms.Comparing the structural maintenance algorithm with the K-means clustering method,t-SNE nonlinear dimension reduction method,multilinear principal component analysis,multilinear discriminant analysis and tensor SIR,the paper shows that the proposed method has obvious advantages in classification accuracy,and has broad development prospect in image recognition and related fields of application.
作者 马少沛 孙庆慧 武雅萱 田茂再 Ma Shaopei;Sun Qinghui;Wu Yaxuan;Tian Maozai
出处 《统计研究》 CSSCI 北大核心 2021年第2期114-134,共21页 Statistical Research
基金 中国人民大学科学研究基金(中央高校基本科研业务费专项资金资助)项目“高维复杂数据充分降维的稳健方法”(20XHN106)
关键词 张量 充分降维 迭代算法 图像识别 Tensor Sufficient Dimension Reduction Iterative Algorithm Image Recognition
  • 相关文献

参考文献2

二级参考文献27

  • 1赵云山,刘焕焕.大数据技术在电力行业的应用研究[J].2013电力行业信息化年会论文集[C].2013. 被引量:1
  • 2Steve Lohr. The Age of Big Data[ N ]. New York Times, 2012 - 02 -11. 被引量:1
  • 3Mckisey Global Institute. Big data : The next frontier for innovation, competition, and productivity[ OL]. 2012 - 08 - 21. 被引量:1
  • 4IBM. What is big Data? [OL]. 2012 -08 -20. 被引量:1
  • 5Panda B, Herbach JS, Beau S, et al. PLANET: Massively parallel learning of tree ensembles with MapReduce [ J ]. PVLDB, 2009,2 (2) :1426 - 1437. 被引量:1
  • 6Kumar V,Andrade H,Gedik B ,et al. DEDUCE:At the intersection of MapReduce and stream processing In : M&nolescu I, Spaccapietra S, Teubner J, KitauregawaM, Leger A, Naumann F, Ailamaki A. Ozcan F, eds. Proc. oftheEDBT. Lausanne: ACM Press, 2010,657 - 662. 被引量:1
  • 7Wieder A, Bhatotia P, Post A, et al. Brief announcement : Modelling MapReduce for optimal execution in the cloud. In: RjchaAW, Guerraouik R, eds. Proe. ofthePODC. Zurich : ACM Press. 2010,408 - 409. 被引量:1
  • 8Big Data. Nature, 2008, 455 (7209) : 1 - 136. 被引量:1
  • 9Dealing with data. Science, 2011, 331(6018) : 639- 806. 被引量:1
  • 10Cohen J, Dolan B, Dunlap M, et al. MAD skills: New analysis practices for big data[ J ]. PVLDB ,2009,2 (2) : 1481 - 1492. 被引量:1

共引文献32

同被引文献38

引证文献4

二级引证文献25

相关作者

内容加载中请稍等...

相关机构

内容加载中请稍等...

相关主题

内容加载中请稍等...

浏览历史

内容加载中请稍等...
;
使用帮助 返回顶部