无监督特征选择是机器学习和数据挖掘中的一种重要的降维技术。然而当前的无监督特征选择方法侧重于从数据的邻接矩阵中学习数据的流形结构,忽视非邻接数据对之间的关联。其次这些方法都假设数据实例具有独立同一性,但现实中的数据样本...无监督特征选择是机器学习和数据挖掘中的一种重要的降维技术。然而当前的无监督特征选择方法侧重于从数据的邻接矩阵中学习数据的流形结构,忽视非邻接数据对之间的关联。其次这些方法都假设数据实例具有独立同一性,但现实中的数据样本其来源是不同的,这样的假设就不成立。此外,在原始数据空间中特征重要性的衡量会受到数据和特征中的噪声影响。基于以上问题,本文提出了潜在多步马尔可夫概率的鲁棒无监督特征选择方法(unsupervised feature selection via multi-step Markov probability and latent representation,MMLRL),其思想是通过最大多步马尔可夫转移概率学习数据流形结构,然后通过对称非负矩阵分解模型学习数据的潜在表示,最后在数据的潜在表示空间中选择特征。同时在6个不同类型的数据集上验证了所提出算法的有效性。展开更多
In this paper,the L_(2,∞)normalization of the weight matrices is used to enhance the robustness and accuracy of the deep neural network(DNN)with Relu as activation functions.It is shown that the L_(2,∞)normalization...In this paper,the L_(2,∞)normalization of the weight matrices is used to enhance the robustness and accuracy of the deep neural network(DNN)with Relu as activation functions.It is shown that the L_(2,∞)normalization leads to large dihedral angles between two adjacent faces of the DNN function graph and hence smoother DNN functions,which reduces over-fitting of the DNN.A global measure is proposed for the robustness of a classification DNN,which is the average radius of the maximal robust spheres with the training samples as centers.A lower bound for the robustness measure in terms of the L_(2,∞)norm is given.Finally,an upper bound for the Rademacher complexity of DNNs with L_(2,∞)normalization is given.An algorithm is given to train DNNs with the L_(2,∞)normalization and numerical experimental results are used to show that the L_(2,∞)normalization is effective in terms of improving the robustness and accuracy.展开更多
As a way of training a single hidden layer feedforward network(SLFN),extreme learning machine(ELM)is rapidly becoming popular due to its efficiency.However,ELM tends to overfitting,which makes the model sensitive to n...As a way of training a single hidden layer feedforward network(SLFN),extreme learning machine(ELM)is rapidly becoming popular due to its efficiency.However,ELM tends to overfitting,which makes the model sensitive to noise and outliers.To solve this problem,L_(2,1)-norm is introduced to ELM and an L_(2,1)-norm robust regularized ELM(L_(2,1)-RRELM)was proposed.L_(2,1)-RRELM gives constant penalties to outliers to reduce their adverse effects by replacing least square loss function with a non-convex loss function.In light of the non-convex feature of L_(2,1)-RRELM,the concave-convex procedure(CCCP)is applied to solve its model.The convergence of L_(2,1)-RRELM is also given to show its robustness.In order to further verify the effectiveness of L_(2,1)-RRELM,it is compared with the three popular extreme learning algorithms based on the artificial dataset and University of California Irvine(UCI)datasets.And each algorithm in different noise environments is tested with two evaluation criterions root mean square error(RMSE)and fitness.The results of the simulation indicate that L_(2,1)-RRELM has smaller RMSE and greater fitness under different noise settings.Numerical analysis shows that L_(2,1)-RRELM has better generalization performance,stronger robustness,and higher anti-noise ability and fitness.展开更多
文摘宽度学习系统(broad learning system,BLS)因其特征提取能力强、计算效率高而被广泛应用于众多领域.然而,目前BLS主要用于单输出回归,当BLS存在多个输出时,BLS无法有效发掘多个输出权重之间的相关性,会导致模型预测性能的下降.鉴于此,通过Frobenius和L_(2,1)矩阵范数的联合约束,提出多输出宽度学习系统(multi-output broad learning system,MOBLS).首先,在原有BLS的基础上构建新的目标函数,将L2损失函数替换为L_(2,1)形式,L_(2)正则化项替换为Frobenius和L_(2,1)两项;然后,利用交替方向乘子法(alternating direction method of multipliers,ADMM)对新目标函数BLS的输出权重优化求解.利用11个公共数据集和1个实际过程数据集验证了所提系统的有效性.
文摘无监督特征选择是机器学习和数据挖掘中的一种重要的降维技术。然而当前的无监督特征选择方法侧重于从数据的邻接矩阵中学习数据的流形结构,忽视非邻接数据对之间的关联。其次这些方法都假设数据实例具有独立同一性,但现实中的数据样本其来源是不同的,这样的假设就不成立。此外,在原始数据空间中特征重要性的衡量会受到数据和特征中的噪声影响。基于以上问题,本文提出了潜在多步马尔可夫概率的鲁棒无监督特征选择方法(unsupervised feature selection via multi-step Markov probability and latent representation,MMLRL),其思想是通过最大多步马尔可夫转移概率学习数据流形结构,然后通过对称非负矩阵分解模型学习数据的潜在表示,最后在数据的潜在表示空间中选择特征。同时在6个不同类型的数据集上验证了所提出算法的有效性。
基金partially supported by NKRDP under Grant No.2018YFA0704705the National Natural Science Foundation of China under Grant No.12288201.
文摘In this paper,the L_(2,∞)normalization of the weight matrices is used to enhance the robustness and accuracy of the deep neural network(DNN)with Relu as activation functions.It is shown that the L_(2,∞)normalization leads to large dihedral angles between two adjacent faces of the DNN function graph and hence smoother DNN functions,which reduces over-fitting of the DNN.A global measure is proposed for the robustness of a classification DNN,which is the average radius of the maximal robust spheres with the training samples as centers.A lower bound for the robustness measure in terms of the L_(2,∞)norm is given.Finally,an upper bound for the Rademacher complexity of DNNs with L_(2,∞)normalization is given.An algorithm is given to train DNNs with the L_(2,∞)normalization and numerical experimental results are used to show that the L_(2,∞)normalization is effective in terms of improving the robustness and accuracy.
基金supported by the National Natural Science Foundation of China(51875457)the Key Research Project of Shaanxi Province(2022GY-050,2022GY-028)+1 种基金the Natural Science Foundation of Shaanxi Province of China(2022JQ-636,2022JQ-705,2021JQ-714)Shaanxi Youth Talent Lifting Plan of Shaanxi Association for Science and Technology(20220129)。
文摘As a way of training a single hidden layer feedforward network(SLFN),extreme learning machine(ELM)is rapidly becoming popular due to its efficiency.However,ELM tends to overfitting,which makes the model sensitive to noise and outliers.To solve this problem,L_(2,1)-norm is introduced to ELM and an L_(2,1)-norm robust regularized ELM(L_(2,1)-RRELM)was proposed.L_(2,1)-RRELM gives constant penalties to outliers to reduce their adverse effects by replacing least square loss function with a non-convex loss function.In light of the non-convex feature of L_(2,1)-RRELM,the concave-convex procedure(CCCP)is applied to solve its model.The convergence of L_(2,1)-RRELM is also given to show its robustness.In order to further verify the effectiveness of L_(2,1)-RRELM,it is compared with the three popular extreme learning algorithms based on the artificial dataset and University of California Irvine(UCI)datasets.And each algorithm in different noise environments is tested with two evaluation criterions root mean square error(RMSE)and fitness.The results of the simulation indicate that L_(2,1)-RRELM has smaller RMSE and greater fitness under different noise settings.Numerical analysis shows that L_(2,1)-RRELM has better generalization performance,stronger robustness,and higher anti-noise ability and fitness.
