摘要
方差缩减算法的主要问题之一是如何选取一个合适的步长。在实践中,手动调整一个最佳的固定步长是很耗时的,所以该文提出将Polya k步长用于随机方差缩减梯度算法(SV RG),得到了一种新的SV RGPolyak算法。对于光滑强凸的目标函数我们证明了SVRG-Polyak算法的线性收敛性。数值实验对比了SVRGPolyak、SVRG和带有BB步长的SVRG(SVRG-BB)3种算法,结果表明SVRG-Polyak算法的有效性。
One of the main problems of the variance reduction algorithm is how to choose an appropriate step size.In practice,it is time-consuming to manually adjust an optimal fixed step size,so this paper proposes to use Polyak step size for the random variance reduction gradient algorithm(SVRG),and a new SVRG-Polyak algorithm is obtained.For the smooth and strongly convex objective function,we prove the linear convergence of the SVRG-Polyak algorithm.Numerical experiments compared SVRG-Polyak,SVRG and SVRG with BB step size(SVRG-BB)three algorithms,and the results show the effectiveness of SVRG-Polyak algorithm.
作者
李蝶
LI Die(College of Science,Hebei University of Technology,Tianjin,300401 China)
出处
《科技资讯》
2021年第16期174-177,共4页
Science & Technology Information
关键词
Polyak步长
方差缩减
强凸
线性收敛
Polyak step size
Variance reduction
Strong convexity
Linear convergence
