针对传统串行聚类集成算法在处理高维海量数据时效率低下的问题,提出基于Spark的并行聚类集成算法SCEA(Spark based Clustering Ensemble Algorithm).首先,通过主成分分析与成对约束结合的方法对算法输入数据进行预处理,达到数据降维并...针对传统串行聚类集成算法在处理高维海量数据时效率低下的问题,提出基于Spark的并行聚类集成算法SCEA(Spark based Clustering Ensemble Algorithm).首先,通过主成分分析与成对约束结合的方法对算法输入数据进行预处理,达到数据降维并去除特征相关性的目的;其次,通过调用不同聚类算法获得基聚类成员后,采用三元组方法通过基聚类成员的簇标签构造出相似度矩阵,并调用层次聚类算法得到最终的聚类结果;最后,在调用MLlib中已有聚类算法的基础上,基于Scala对SCEA算法进行了实现.将SCEA与同类算法在多组数据集下进行对比测试,实验结果表明:总体上SCEA不仅较已有算法在准确率方面有所提高,并且通过分析运行时间、加速比以及可扩展性3个性能指标,证明了SCEA在算法性能上的优越性.展开更多
On-line transient stability analysis of a power grid is crucial in determining whether the power grid will traverse to a steady state stable operating point after a disturbance. The transient stability analysis involv...On-line transient stability analysis of a power grid is crucial in determining whether the power grid will traverse to a steady state stable operating point after a disturbance. The transient stability analysis involves computing the solutions of the algebraic equations modeling the grid network and the ordinary differential equations modeling the dynamics of the electrical components like synchronous generators, exciters, governors, etc., of the grid in near real-time. In this research, we investigate the use of time-parallel approach in particular the Parareal algorithm implementation on Graphical Processing Unit using Compute Unified Device Architecture to compute solutions of ordinary differential equations. The numerical solution accuracy and computation time of the Parareal algorithm executing on the GPU are demonstrated on the single machine infinite bus test system. Two types of dynamic model of the single synchronous generator namely the classical and detailed models are studied. The numerical solutions of the ordinary differential equations computed by the Parareal algorithm are compared to that computed using the modified Euler’s method demonstrating the accuracy of the Parareal algorithm executing on GPU. Simulations are performed with varying numerical integration time steps, and the suitability of Parareal algorithm in computing near real-time solutions of ordinary different equations is presented. A speedup of 25× and 31× is achieved with the Parareal algorithm for classical and detailed dynamic models of the synchronous generator respectively compared to the sequential modified Euler’s method. The weak scaling efficiency of the Parareal algorithm when required to solve a large number of ordinary differential equations at each time step due to the increase in sequential computations and associated memory transfer latency between the CPU and GPU is discussed.展开更多
文摘针对传统串行聚类集成算法在处理高维海量数据时效率低下的问题,提出基于Spark的并行聚类集成算法SCEA(Spark based Clustering Ensemble Algorithm).首先,通过主成分分析与成对约束结合的方法对算法输入数据进行预处理,达到数据降维并去除特征相关性的目的;其次,通过调用不同聚类算法获得基聚类成员后,采用三元组方法通过基聚类成员的簇标签构造出相似度矩阵,并调用层次聚类算法得到最终的聚类结果;最后,在调用MLlib中已有聚类算法的基础上,基于Scala对SCEA算法进行了实现.将SCEA与同类算法在多组数据集下进行对比测试,实验结果表明:总体上SCEA不仅较已有算法在准确率方面有所提高,并且通过分析运行时间、加速比以及可扩展性3个性能指标,证明了SCEA在算法性能上的优越性.
文摘On-line transient stability analysis of a power grid is crucial in determining whether the power grid will traverse to a steady state stable operating point after a disturbance. The transient stability analysis involves computing the solutions of the algebraic equations modeling the grid network and the ordinary differential equations modeling the dynamics of the electrical components like synchronous generators, exciters, governors, etc., of the grid in near real-time. In this research, we investigate the use of time-parallel approach in particular the Parareal algorithm implementation on Graphical Processing Unit using Compute Unified Device Architecture to compute solutions of ordinary differential equations. The numerical solution accuracy and computation time of the Parareal algorithm executing on the GPU are demonstrated on the single machine infinite bus test system. Two types of dynamic model of the single synchronous generator namely the classical and detailed models are studied. The numerical solutions of the ordinary differential equations computed by the Parareal algorithm are compared to that computed using the modified Euler’s method demonstrating the accuracy of the Parareal algorithm executing on GPU. Simulations are performed with varying numerical integration time steps, and the suitability of Parareal algorithm in computing near real-time solutions of ordinary different equations is presented. A speedup of 25× and 31× is achieved with the Parareal algorithm for classical and detailed dynamic models of the synchronous generator respectively compared to the sequential modified Euler’s method. The weak scaling efficiency of the Parareal algorithm when required to solve a large number of ordinary differential equations at each time step due to the increase in sequential computations and associated memory transfer latency between the CPU and GPU is discussed.
