In this work, a gradient method with momentum for BP neural networks is considered. The momentum coefficient is chosen in an adaptive manner to accelerate and stabilize the learning procedure of the network weights. C...In this work, a gradient method with momentum for BP neural networks is considered. The momentum coefficient is chosen in an adaptive manner to accelerate and stabilize the learning procedure of the network weights. Corresponding convergence results are proved.展开更多
As an emerging joint learning model,federated learning is a promising way to combine model parameters of different users for training and inference without collecting users’original data.However,a practical and effic...As an emerging joint learning model,federated learning is a promising way to combine model parameters of different users for training and inference without collecting users’original data.However,a practical and efficient solution has not been established in previous work due to the absence of efficient matrix computation and cryptography schemes in the privacy-preserving federated learning model,especially in partially homomorphic cryptosystems.In this paper,we propose a Practical and Efficient Privacy-preserving Federated Learning(PEPFL)framework.First,we present a lifted distributed ElGamal cryptosystem for federated learning,which can solve the multi-key problem in federated learning.Secondly,we develop a Practical Partially Single Instruction Multiple Data(PSIMD)parallelism scheme that can encode a plaintext matrix into single plaintext for encryption,improving the encryption efficiency and reducing the communication cost in partially homomorphic cryptosystem.In addition,based on the Convolutional Neural Network(CNN)and the designed cryptosystem,a novel privacy-preserving federated learning framework is designed by using Momentum Gradient Descent(MGD).Finally,we evaluate the security and performance of PEPFL.The experiment results demonstrate that the scheme is practicable,effective,and secure with low communication and computation costs.展开更多
In this paper, a gradient method with momentum for sigma-pi-sigma neural networks (SPSNN) is considered in order to accelerate the convergence of the learning procedure for the network weights. The momentum coefficien...In this paper, a gradient method with momentum for sigma-pi-sigma neural networks (SPSNN) is considered in order to accelerate the convergence of the learning procedure for the network weights. The momentum coefficient is chosen in an adaptive manner, and the corresponding weak convergence and strong convergence results are proved.展开更多
To solve the problem of large steady state residual error of momentum constant modulus algorithm (CMA) blind equalization, a momentum CMA blind equalization controlled by energy steady state was proposed. The energy o...To solve the problem of large steady state residual error of momentum constant modulus algorithm (CMA) blind equalization, a momentum CMA blind equalization controlled by energy steady state was proposed. The energy of the equalizer weights is estimated during the updating process. According to the adaptive filtering theory, the energy of the equalizer weights reaches to the steady state after the algorithm is converged, and then the momentum can be set to 0 when the energy change rate is less than the threshold, which can avoid the additional gradient noise caused by momentum and further improve the convergence precision of the algorithm. The proposed algorithm takes advantage of momentum to quicken the convergence rate and to avoid the local minimum in the cost function to some extent;meanwhile, it has the same convergence precision with CMA. Computer simulation results show that, compared with CMA, momentum CMA (MCMA) and adaptive momentum CMA (AMCMA) blind equalization, the proposed algorithm has the fastest convergence rate and the same steady state residual error with CMA.展开更多
We consider small vortices, such as tornadoes, dust devils, whirlpools, and small hurricanes at low latitudes, for which the Coriolis force can be neglected. Such vortices are (at least approximately) cylindrically sy...We consider small vortices, such as tornadoes, dust devils, whirlpools, and small hurricanes at low latitudes, for which the Coriolis force can be neglected. Such vortices are (at least approximately) cylindrically symmetrical about a vertical axis through the center of a calm central region or eye of radius . In the region fluid (gas or liquid) circulates about the eye with speed . We take to be the outer periphery of the vortex, where the fluid speed is reduced to that of the surrounding wind field (in the cases of tornadoes, dust devils, and small hurricanes at low latitudes) or deemed negligible (in the case of whirlpools). If , angular momentum is conserved within the fluid itself;if , angular momentum must be exchanged with Earth to ensure conservation of total angular momentum. We derive the steepness and upper limit of the pressure gradients in vortices. We then discuss the power and energy of vortices. We compare the kinetic energy of atmospheric vortices and the power required to maintain them against frictional dissipation with the same quantities for Earth’s atmosphere as a whole. We explain why the kinetic energy of atmospheric vortices must be replaced on much shorter timescales than is the case for Earth’s atmosphere as a whole. Brief comparisons of cyclostrophic flow with geostrophic and friction-balanced flows are then provided. We then consider an analogy that might be drawn, at least to some extent, with gravitational systems, considering mainly spherically-symmetrical and cylindrically-symmetrical ones. Generation of kinetic energy at the expense of potential energy in fluid vortices, in geostrophic and friction-balanced flows, and in gravitational systems is then briefly discussed. We explain the variations of pressure and gravitational gradients corresponding to generation of kinetic energy exceeding, equaling, and falling short of frictional dissipation. In the Appendix, we describe a simple method for maximizing power extraction from environmental fluid (air or water) flows. In summary展开更多
A new type of power-exponent-phase vortex-like beams with both quadratic and cubic azimuthal phase gradients is investigated in this work.The intensity and orbital angular momentum(OAM)density distributions are notice...A new type of power-exponent-phase vortex-like beams with both quadratic and cubic azimuthal phase gradients is investigated in this work.The intensity and orbital angular momentum(OAM)density distributions are noticeably different when the phase gradient increases or decreases along the azimuth angle,while the orthogonality and total OAM remain constant.The characteristics of the optical field undergo a significant change when the phase shifts from linear to nonlinear,with the variation of the power index having little impact on the beam characteristics under nonlinear phase conditions.These characteristics provide new ideas for applications such as particle manipulation,optical communications,and OAM encryption.展开更多
Ⅰ. INTRODUCTIONIt is very well known from the observations that some atmospheric motions are accompanied by jets in the boundary layer, for example, breezes and circulations in the mountain valleys (Gutman, 1969); n... Ⅰ. INTRODUCTIONIt is very well known from the observations that some atmospheric motions are accompanied by jets in the boundary layer, for example, breezes and circulations in the mountain valleys (Gutman, 1969); nocturnal increasing of wind (Byzova et al., 1989); cross-equatorial flow during the summer Indian monsoon (Das, 1986) and others. One of the important questions concerning a mathematical modelling of such motions is the problem of the turbulent closure of the equations set which describes the jet dynamics. It is still popular to use for the momentum turbulent flow (u＇w＇) a closure, based within the framework of K-theory on the Boussinesq hypothesis展开更多
Gradient descent(GD)algorithm is the widely used optimisation method in training machine learning and deep learning models.In this paper,based on GD,Polyak’s momentum(PM),and Nesterov accelerated gradient(NAG),we giv...Gradient descent(GD)algorithm is the widely used optimisation method in training machine learning and deep learning models.In this paper,based on GD,Polyak’s momentum(PM),and Nesterov accelerated gradient(NAG),we give the convergence of the algorithms from an ini-tial value to the optimal value of an objective function in simple quadratic form.Based on the convergence property of the quadratic function,two sister sequences of NAG’s iteration and par-allel tangent methods in neural networks,the three-step accelerated gradient(TAG)algorithm is proposed,which has three sequences other than two sister sequences.To illustrate the perfor-mance of this algorithm,we compare the proposed algorithm with the three other algorithms in quadratic function,high-dimensional quadratic functions,and nonquadratic function.Then we consider to combine the TAG algorithm to the backpropagation algorithm and the stochastic gradient descent algorithm in deep learning.For conveniently facilitate the proposed algorithms,we rewite the R package‘neuralnet’and extend it to‘supneuralnet’.All kinds of deep learning algorithms in this paper are included in‘supneuralnet’package.Finally,we show our algorithms are superior to other algorithms in four case studies.展开更多
A vector potential of a magnetic field in Lagrangian is defined as the necessary partial solution of a inhomogeneous differential equation. The ＂gradient transformation＂ is an addition of arbitrary general solution ...A vector potential of a magnetic field in Lagrangian is defined as the necessary partial solution of a inhomogeneous differential equation. The ＂gradient transformation＂ is an addition of arbitrary general solution of the corresponding homogeneous equation that does not change the Lagrange equations. When dynamics is described by momenta and coordinates, this transformation is not the vector potential modification, which does not change expressions for other physical quantities, but a canonical transformation of momentum, which changes expressions for all fimctions of momentum, not changing the Poisson brackets, and, hence, the integrals of motion. The generating function of this transformation must reverse sign under the time-charge reversal. In quantum mechanics the unitary transformation corresponds to this canonical transformation. It also does not change the commutation relations. The phase of this unitary operator also must reverse sign under the time-charge reversal. Examples of necessary vector potentials for some magnetic fields are presented.展开更多
基金National Natural Science Foundation of China (10471017)Zhejiang Provincial Natural Science Foundation (Y606009)
文摘In this work, a gradient method with momentum for BP neural networks is considered. The momentum coefficient is chosen in an adaptive manner to accelerate and stabilize the learning procedure of the network weights. Corresponding convergence results are proved.
基金supported by the National Natural Science Foundation of China under Grant No.U19B2021the Key Research and Development Program of Shaanxi under Grant No.2020ZDLGY08-04+1 种基金the Key Technologies R&D Program of He’nan Province under Grant No.212102210084the Innovation Scientists and Technicians Troop Construction Projects of Henan Province.
文摘As an emerging joint learning model,federated learning is a promising way to combine model parameters of different users for training and inference without collecting users’original data.However,a practical and efficient solution has not been established in previous work due to the absence of efficient matrix computation and cryptography schemes in the privacy-preserving federated learning model,especially in partially homomorphic cryptosystems.In this paper,we propose a Practical and Efficient Privacy-preserving Federated Learning(PEPFL)framework.First,we present a lifted distributed ElGamal cryptosystem for federated learning,which can solve the multi-key problem in federated learning.Secondly,we develop a Practical Partially Single Instruction Multiple Data(PSIMD)parallelism scheme that can encode a plaintext matrix into single plaintext for encryption,improving the encryption efficiency and reducing the communication cost in partially homomorphic cryptosystem.In addition,based on the Convolutional Neural Network(CNN)and the designed cryptosystem,a novel privacy-preserving federated learning framework is designed by using Momentum Gradient Descent(MGD).Finally,we evaluate the security and performance of PEPFL.The experiment results demonstrate that the scheme is practicable,effective,and secure with low communication and computation costs.
文摘In this paper, a gradient method with momentum for sigma-pi-sigma neural networks (SPSNN) is considered in order to accelerate the convergence of the learning procedure for the network weights. The momentum coefficient is chosen in an adaptive manner, and the corresponding weak convergence and strong convergence results are proved.
文摘To solve the problem of large steady state residual error of momentum constant modulus algorithm (CMA) blind equalization, a momentum CMA blind equalization controlled by energy steady state was proposed. The energy of the equalizer weights is estimated during the updating process. According to the adaptive filtering theory, the energy of the equalizer weights reaches to the steady state after the algorithm is converged, and then the momentum can be set to 0 when the energy change rate is less than the threshold, which can avoid the additional gradient noise caused by momentum and further improve the convergence precision of the algorithm. The proposed algorithm takes advantage of momentum to quicken the convergence rate and to avoid the local minimum in the cost function to some extent;meanwhile, it has the same convergence precision with CMA. Computer simulation results show that, compared with CMA, momentum CMA (MCMA) and adaptive momentum CMA (AMCMA) blind equalization, the proposed algorithm has the fastest convergence rate and the same steady state residual error with CMA.
文摘We consider small vortices, such as tornadoes, dust devils, whirlpools, and small hurricanes at low latitudes, for which the Coriolis force can be neglected. Such vortices are (at least approximately) cylindrically symmetrical about a vertical axis through the center of a calm central region or eye of radius . In the region fluid (gas or liquid) circulates about the eye with speed . We take to be the outer periphery of the vortex, where the fluid speed is reduced to that of the surrounding wind field (in the cases of tornadoes, dust devils, and small hurricanes at low latitudes) or deemed negligible (in the case of whirlpools). If , angular momentum is conserved within the fluid itself;if , angular momentum must be exchanged with Earth to ensure conservation of total angular momentum. We derive the steepness and upper limit of the pressure gradients in vortices. We then discuss the power and energy of vortices. We compare the kinetic energy of atmospheric vortices and the power required to maintain them against frictional dissipation with the same quantities for Earth’s atmosphere as a whole. We explain why the kinetic energy of atmospheric vortices must be replaced on much shorter timescales than is the case for Earth’s atmosphere as a whole. Brief comparisons of cyclostrophic flow with geostrophic and friction-balanced flows are then provided. We then consider an analogy that might be drawn, at least to some extent, with gravitational systems, considering mainly spherically-symmetrical and cylindrically-symmetrical ones. Generation of kinetic energy at the expense of potential energy in fluid vortices, in geostrophic and friction-balanced flows, and in gravitational systems is then briefly discussed. We explain the variations of pressure and gravitational gradients corresponding to generation of kinetic energy exceeding, equaling, and falling short of frictional dissipation. In the Appendix, we describe a simple method for maximizing power extraction from environmental fluid (air or water) flows. In summary
基金supported by the Guangdong Major Project of Basic and Applied Basic Research(No.2020B0301030009)National Natural Science Foundation of China(Nos.61935013,62375181,and 61975133)+1 种基金Science and Technology Innovation Commission of Shenzhen,Shenzhen Peacock Plan(Nos.KQTD20170330110444030 and JCYJ20200109114018750)Shenzhen University(No.2019075)。
文摘A new type of power-exponent-phase vortex-like beams with both quadratic and cubic azimuthal phase gradients is investigated in this work.The intensity and orbital angular momentum(OAM)density distributions are noticeably different when the phase gradient increases or decreases along the azimuth angle,while the orthogonality and total OAM remain constant.The characteristics of the optical field undergo a significant change when the phase shifts from linear to nonlinear,with the variation of the power index having little impact on the beam characteristics under nonlinear phase conditions.These characteristics provide new ideas for applications such as particle manipulation,optical communications,and OAM encryption.
文摘 Ⅰ. INTRODUCTIONIt is very well known from the observations that some atmospheric motions are accompanied by jets in the boundary layer, for example, breezes and circulations in the mountain valleys (Gutman, 1969); nocturnal increasing of wind (Byzova et al., 1989); cross-equatorial flow during the summer Indian monsoon (Das, 1986) and others. One of the important questions concerning a mathematical modelling of such motions is the problem of the turbulent closure of the equations set which describes the jet dynamics. It is still popular to use for the momentum turbulent flow (u＇w＇) a closure, based within the framework of K-theory on the Boussinesq hypothesis
基金This work was supported by National Natural Science Foun-dation of China(11271136,81530086)Program of Shanghai Subject Chief Scientist(14XD1401600)the 111 Project of China(No.B14019).
文摘Gradient descent(GD)algorithm is the widely used optimisation method in training machine learning and deep learning models.In this paper,based on GD,Polyak’s momentum(PM),and Nesterov accelerated gradient(NAG),we give the convergence of the algorithms from an ini-tial value to the optimal value of an objective function in simple quadratic form.Based on the convergence property of the quadratic function,two sister sequences of NAG’s iteration and par-allel tangent methods in neural networks,the three-step accelerated gradient(TAG)algorithm is proposed,which has three sequences other than two sister sequences.To illustrate the perfor-mance of this algorithm,we compare the proposed algorithm with the three other algorithms in quadratic function,high-dimensional quadratic functions,and nonquadratic function.Then we consider to combine the TAG algorithm to the backpropagation algorithm and the stochastic gradient descent algorithm in deep learning.For conveniently facilitate the proposed algorithms,we rewite the R package‘neuralnet’and extend it to‘supneuralnet’.All kinds of deep learning algorithms in this paper are included in‘supneuralnet’package.Finally,we show our algorithms are superior to other algorithms in four case studies.
文摘A vector potential of a magnetic field in Lagrangian is defined as the necessary partial solution of a inhomogeneous differential equation. The ＂gradient transformation＂ is an addition of arbitrary general solution of the corresponding homogeneous equation that does not change the Lagrange equations. When dynamics is described by momenta and coordinates, this transformation is not the vector potential modification, which does not change expressions for other physical quantities, but a canonical transformation of momentum, which changes expressions for all fimctions of momentum, not changing the Poisson brackets, and, hence, the integrals of motion. The generating function of this transformation must reverse sign under the time-charge reversal. In quantum mechanics the unitary transformation corresponds to this canonical transformation. It also does not change the commutation relations. The phase of this unitary operator also must reverse sign under the time-charge reversal. Examples of necessary vector potentials for some magnetic fields are presented.
