Deep metric learning(DML)has achieved great results on visual understanding tasks by seamlessly integrating conventional metric learning with deep neural networks.Existing deep metric learning methods focus on designi...Deep metric learning(DML)has achieved great results on visual understanding tasks by seamlessly integrating conventional metric learning with deep neural networks.Existing deep metric learning methods focus on designing pair-based distance loss to decrease intra-class distance while increasing interclass distance.However,these methods fail to preserve the geometric structure of data in the embedding space,which leads to the spatial structure shift across mini-batches and may slow down the convergence of embedding learning.To alleviate these issues,by assuming that the input data is embedded in a lower-dimensional sub-manifold,we propose a novel deep Riemannian metric learning(DRML)framework that exploits the non-Euclidean geometric structural information.Considering that the curvature information of data measures how much the Riemannian(nonEuclidean)metric deviates from the Euclidean metric,we leverage geometry flow,which is called a geometric evolution equation,to characterize the relation between the Riemannian metric and its curvature.Our DRML not only regularizes the local neighborhoods connection of the embeddings at the hidden layer but also adapts the embeddings to preserve the geometric structure of the data.On several benchmark datasets,the proposed DRML outperforms all existing methods and these results demonstrate its effectiveness.展开更多
The privacy and security of data are recently research hotspots and challenges.For this issue,an adaptive scheme of distributed learning based on homomorphic encryption and blockchain is proposed.Specifically,in the f...The privacy and security of data are recently research hotspots and challenges.For this issue,an adaptive scheme of distributed learning based on homomorphic encryption and blockchain is proposed.Specifically,in the form of homomorphic encryption,the computing party iteratively aggregates the learning models from distributed participants,so that the privacy of both the data and model is ensured.Moreover,the aggregations are recorded and verified by blockchain,which prevents attacks from malicious nodes and guarantees the reliability of learning.For these sophisticated privacy and security technologies,the computation cost and energy consumption in both the encrypted learning and consensus reaching are analyzed,based on which a joint optimization of computation resources allocation and adaptive aggregation to minimize loss function is established with the realistic solution followed.Finally,the simulations and analysis evaluate the performance of the proposed scheme.展开更多
With the emergence of various intelligent applications,machine learning technologies face lots of challenges including large-scale models,application oriented real-time dataset and limited capabilities of nodes in pra...With the emergence of various intelligent applications,machine learning technologies face lots of challenges including large-scale models,application oriented real-time dataset and limited capabilities of nodes in practice.Therefore,distributed machine learning(DML) and semi-supervised learning methods which help solve these problems have been addressed in both academia and industry.In this paper,the semi-supervised learning method and the data parallelism DML framework are combined.The pseudo-label based local loss function for each distributed node is studied,and the stochastic gradient descent(SGD) based distributed parameter update principle is derived.A demo that implements the pseudo-label based semi-supervised learning in the DML framework is conducted,and the CIFAR-10 dataset for target classification is used to evaluate the performance.Experimental results confirm the convergence and the accuracy of the model using the pseudo-label based semi-supervised learning in the DML framework.Given the proportion of the pseudo-label dataset is 20%,the accuracy of the model is over 90% when the value of local parameter update steps between two global aggregations is less than 5.Besides,fixing the global aggregations interval to 3,the model converges with acceptable performance degradation when the proportion of the pseudo-label dataset varies from 20% to 80%.展开更多
基金supported in part by the Young Elite Scientists Sponsorship Program by CAST(2022QNRC001)the National Natural Science Foundation of China(61621003,62101136)+2 种基金Natural Science Foundation of Shanghai(21ZR1403600)Shanghai Municipal Science and Technology Major Project(2018SHZDZX01)ZJLab,and Shanghai Municipal of Science and Technology Project(20JC1419500)。
文摘Deep metric learning(DML)has achieved great results on visual understanding tasks by seamlessly integrating conventional metric learning with deep neural networks.Existing deep metric learning methods focus on designing pair-based distance loss to decrease intra-class distance while increasing interclass distance.However,these methods fail to preserve the geometric structure of data in the embedding space,which leads to the spatial structure shift across mini-batches and may slow down the convergence of embedding learning.To alleviate these issues,by assuming that the input data is embedded in a lower-dimensional sub-manifold,we propose a novel deep Riemannian metric learning(DRML)framework that exploits the non-Euclidean geometric structural information.Considering that the curvature information of data measures how much the Riemannian(nonEuclidean)metric deviates from the Euclidean metric,we leverage geometry flow,which is called a geometric evolution equation,to characterize the relation between the Riemannian metric and its curvature.Our DRML not only regularizes the local neighborhoods connection of the embeddings at the hidden layer but also adapts the embeddings to preserve the geometric structure of the data.On several benchmark datasets,the proposed DRML outperforms all existing methods and these results demonstrate its effectiveness.
基金Supported by the National Natural Science Foundation of China(No.62171062)Foundation of Beijing Municipal Commission of Education(No.KM202010005017,KM202110005021)Beijing Natural Science Foundation(No.L211002)。
文摘The privacy and security of data are recently research hotspots and challenges.For this issue,an adaptive scheme of distributed learning based on homomorphic encryption and blockchain is proposed.Specifically,in the form of homomorphic encryption,the computing party iteratively aggregates the learning models from distributed participants,so that the privacy of both the data and model is ensured.Moreover,the aggregations are recorded and verified by blockchain,which prevents attacks from malicious nodes and guarantees the reliability of learning.For these sophisticated privacy and security technologies,the computation cost and energy consumption in both the encrypted learning and consensus reaching are analyzed,based on which a joint optimization of computation resources allocation and adaptive aggregation to minimize loss function is established with the realistic solution followed.Finally,the simulations and analysis evaluate the performance of the proposed scheme.
基金Supported by the National Key R&D Program of China(No.2020YFC1807904)the Natural Science Foundation of Beijing Municipality(No.L192002)the National Natural Science Foundation of China(No.U1633115)。
文摘With the emergence of various intelligent applications,machine learning technologies face lots of challenges including large-scale models,application oriented real-time dataset and limited capabilities of nodes in practice.Therefore,distributed machine learning(DML) and semi-supervised learning methods which help solve these problems have been addressed in both academia and industry.In this paper,the semi-supervised learning method and the data parallelism DML framework are combined.The pseudo-label based local loss function for each distributed node is studied,and the stochastic gradient descent(SGD) based distributed parameter update principle is derived.A demo that implements the pseudo-label based semi-supervised learning in the DML framework is conducted,and the CIFAR-10 dataset for target classification is used to evaluate the performance.Experimental results confirm the convergence and the accuracy of the model using the pseudo-label based semi-supervised learning in the DML framework.Given the proportion of the pseudo-label dataset is 20%,the accuracy of the model is over 90% when the value of local parameter update steps between two global aggregations is less than 5.Besides,fixing the global aggregations interval to 3,the model converges with acceptable performance degradation when the proportion of the pseudo-label dataset varies from 20% to 80%.
