In this paper, a new trust region method with simple model for solving large-scale unconstrained nonlinear optimization is proposed. By employing the generalized weak quasi-Newton equations, we derive several schemes ...In this paper, a new trust region method with simple model for solving large-scale unconstrained nonlinear optimization is proposed. By employing the generalized weak quasi-Newton equations, we derive several schemes to construct variants of scalar matrices as the Hessian approximation used in the trust region subproblem. Under some reasonable conditions, global convergence of the proposed algorithm is established in the trust region framework. The numerical experiments on solving the test problems with dimensions from 50 to 20,000 in the CUTEr library are reported to show efficiency of the algorithm.展开更多
In this paper,by designing a normalized nonmonotone search strategy with the BarzilaiBorwein-type step-size,a novel local minimax method(LMM),which is a globally convergent iterative method,is proposed and analyzed to...In this paper,by designing a normalized nonmonotone search strategy with the BarzilaiBorwein-type step-size,a novel local minimax method(LMM),which is a globally convergent iterative method,is proposed and analyzed to find multiple(unstable)saddle points of nonconvex functionals in Hilbert spaces.Compared to traditional LMMs with monotone search strategies,this approach,which does not require strict decrease of the objective functional value at each iterative step,is observed to converge faster with less computations.Firstly,based on a normalized iterative scheme coupled with a local peak selection that pulls the iterative point back onto the solution submanifold,by generalizing the Zhang-Hager(ZH)search strategy in the optimization theory to the LMM framework,a kind of normalized ZH-type nonmonotone step-size search strategy is introduced,and then a novel nonmonotone LMM is constructed.Its feasibility and global convergence results are rigorously carried out under the relaxation of the monotonicity for the functional at the iterative sequences.Secondly,in order to speed up the convergence of the nonmonotone LMM,a globally convergent Barzilai-Borwein-type LMM(GBBLMM)is presented by explicitly constructing the Barzilai-Borwein-type step-size as a trial step-size of the normalized ZH-type nonmonotone step-size search strategy in each iteration.Finally,the GBBLMM algorithm is implemented to find multiple unstable solutions of two classes of semilinear elliptic boundary value problems with variational structures:one is the semilinear elliptic equations with the homogeneous Dirichlet boundary condition and another is the linear elliptic equations with semilinear Neumann boundary conditions.Extensive numerical results indicate that our approach is very effective and speeds up the LMMs significantly.展开更多
In this study, we propose a linearized proximal alternating direction method with variable stepsize for solving total variation image reconstruction problems. Our method uses a linearized technique and the proximal fu...In this study, we propose a linearized proximal alternating direction method with variable stepsize for solving total variation image reconstruction problems. Our method uses a linearized technique and the proximal function such that the closed form solutions of the subproblem can be easily derived.In the subproblem, we apply a variable stepsize, that is like Barzilai-Borwein stepsize, to accelerate the algorithm. Numerical results with parallel magnetic resonance imaging demonstrate the efficiency of the proposed algorithm.展开更多
Many machine learning problems can be formulated as minimizing the sum of a function and a non-smooth regularization term.Proximal stochastic gradient methods are popular for solving such composite optimization proble...Many machine learning problems can be formulated as minimizing the sum of a function and a non-smooth regularization term.Proximal stochastic gradient methods are popular for solving such composite optimization problems.We propose a minibatch proximal stochastic recursive gradient algorithm SRG-DBB,which incorporates the diagonal Barzilai–Borwein(DBB)stepsize strategy to capture the local geometry of the problem.The linear convergence and complexity of SRG-DBB are analyzed for strongly convex functions.We further establish the linear convergence of SRGDBB under the non-strong convexity condition.Moreover,it is proved that SRG-DBB converges sublinearly in the convex case.Numerical experiments on standard data sets indicate that the performance of SRG-DBB is better than or comparable to the proximal stochastic recursive gradient algorithm with best-tuned scalar stepsizes or BB stepsizes.Furthermore,SRG-DBB is superior to some advanced mini-batch proximal stochastic gradient methods.展开更多
In this paper,we propose a new nonmonotone trust region Barzilai-Borwein(BB for short)method for solving unconstrained optimization problems.The proposed method is given by a novel combination of a modified Metropolis...In this paper,we propose a new nonmonotone trust region Barzilai-Borwein(BB for short)method for solving unconstrained optimization problems.The proposed method is given by a novel combination of a modified Metropolis criterion,BB-stepsize and trust region method.The new method uses the reciprocal of BB-stepsize to approximate the Hessian matrix of the objective function in the trust region subproblems,and accepts some bad solutions according to the modified Metropolis criterion based on simulated annealing idea.Under some suitable assumptions,the global convergence of the new method is established.Some preliminary numerical results indicate that,the new method is more efficient compared with the existing trust region BB method.展开更多
In this paper,we develop an active set identification technique.By means of the active set technique,we present an active set adaptive monotone projected Barzilai-Borwein method(ASAMPBB)for solving nonnegative matrix ...In this paper,we develop an active set identification technique.By means of the active set technique,we present an active set adaptive monotone projected Barzilai-Borwein method(ASAMPBB)for solving nonnegative matrix factorization(NMF)based on the alternating nonnegative least squares framework,in which the Barzilai-Borwein(BB)step sizes can be adaptively picked to get meaningful convergence rate improvements.To get optimal step size,we take into account of the curvature information.In addition,the larger step size technique is exploited to accelerate convergence of the proposed method.The global convergence of the proposed method is analysed under mild assumption.Finally,the results of the numerical experiments on both synthetic and real-world datasets show that the proposed method is effective.展开更多
基金supported by National Natural Science Foundation of China (Grant Nos. 11571178, 11401308, 11371197 and 11471145)the National Science Foundation of USA (Grant No. 1522654)a Project Funded by the Priority Academic Program Development of Jiangsu Higher Education Institutions
文摘In this paper, a new trust region method with simple model for solving large-scale unconstrained nonlinear optimization is proposed. By employing the generalized weak quasi-Newton equations, we derive several schemes to construct variants of scalar matrices as the Hessian approximation used in the trust region subproblem. Under some reasonable conditions, global convergence of the proposed algorithm is established in the trust region framework. The numerical experiments on solving the test problems with dimensions from 50 to 20,000 in the CUTEr library are reported to show efficiency of the algorithm.
基金supported by the NSFC(Grant Nos.12171148,11771138)the NSFC(Grant Nos.12101252,11971007)+2 种基金the NSFC(Grant No.11901185)the National Key R&D Program of China(Grant No.2021YFA1001300)by the Fundamental Research Funds for the Central Universities(Grant No.531118010207).
文摘In this paper,by designing a normalized nonmonotone search strategy with the BarzilaiBorwein-type step-size,a novel local minimax method(LMM),which is a globally convergent iterative method,is proposed and analyzed to find multiple(unstable)saddle points of nonconvex functionals in Hilbert spaces.Compared to traditional LMMs with monotone search strategies,this approach,which does not require strict decrease of the objective functional value at each iterative step,is observed to converge faster with less computations.Firstly,based on a normalized iterative scheme coupled with a local peak selection that pulls the iterative point back onto the solution submanifold,by generalizing the Zhang-Hager(ZH)search strategy in the optimization theory to the LMM framework,a kind of normalized ZH-type nonmonotone step-size search strategy is introduced,and then a novel nonmonotone LMM is constructed.Its feasibility and global convergence results are rigorously carried out under the relaxation of the monotonicity for the functional at the iterative sequences.Secondly,in order to speed up the convergence of the nonmonotone LMM,a globally convergent Barzilai-Borwein-type LMM(GBBLMM)is presented by explicitly constructing the Barzilai-Borwein-type step-size as a trial step-size of the normalized ZH-type nonmonotone step-size search strategy in each iteration.Finally,the GBBLMM algorithm is implemented to find multiple unstable solutions of two classes of semilinear elliptic boundary value problems with variational structures:one is the semilinear elliptic equations with the homogeneous Dirichlet boundary condition and another is the linear elliptic equations with semilinear Neumann boundary conditions.Extensive numerical results indicate that our approach is very effective and speeds up the LMMs significantly.
基金supported in part by the National Natural Science Foundation of China(11361018,11461015)Guangxi Natural Science Foundation(2014GXNSFFA118001)+3 种基金Guangxi Key Laboratory of Automatic Detecting Technology and Instruments(YQ15112,YQ16112)Guilin Science and Technology Project(20140127-2)the Innovation Project of Guangxi Graduate Education and Innovation Project of GUET Graduate Education(YJCXB201502)Guangxi Key Laboratory of Cryptography and Information Security(GCIS201624)
文摘In this study, we propose a linearized proximal alternating direction method with variable stepsize for solving total variation image reconstruction problems. Our method uses a linearized technique and the proximal function such that the closed form solutions of the subproblem can be easily derived.In the subproblem, we apply a variable stepsize, that is like Barzilai-Borwein stepsize, to accelerate the algorithm. Numerical results with parallel magnetic resonance imaging demonstrate the efficiency of the proposed algorithm.
基金the National Natural Science Foundation of China(Nos.11671116,11701137,12071108,11991020,11991021 and 12021001)the Major Research Plan of the NSFC(No.91630202)+1 种基金the Strategic Priority Research Program of Chinese Academy of Sciences(No.XDA27000000)the Natural Science Foundation of Hebei Province(No.A2021202010)。
文摘Many machine learning problems can be formulated as minimizing the sum of a function and a non-smooth regularization term.Proximal stochastic gradient methods are popular for solving such composite optimization problems.We propose a minibatch proximal stochastic recursive gradient algorithm SRG-DBB,which incorporates the diagonal Barzilai–Borwein(DBB)stepsize strategy to capture the local geometry of the problem.The linear convergence and complexity of SRG-DBB are analyzed for strongly convex functions.We further establish the linear convergence of SRGDBB under the non-strong convexity condition.Moreover,it is proved that SRG-DBB converges sublinearly in the convex case.Numerical experiments on standard data sets indicate that the performance of SRG-DBB is better than or comparable to the proximal stochastic recursive gradient algorithm with best-tuned scalar stepsizes or BB stepsizes.Furthermore,SRG-DBB is superior to some advanced mini-batch proximal stochastic gradient methods.
基金supported by the National Natural Science Foundation of China(Nos.12071398,11671125,11571074,61977017)the Natural Science Foundation of Hunan Province(No.2020JJ4567)the Key Scientific Research Found of Hunan Education Department(No.20A097)。
文摘In this paper,we propose a new nonmonotone trust region Barzilai-Borwein(BB for short)method for solving unconstrained optimization problems.The proposed method is given by a novel combination of a modified Metropolis criterion,BB-stepsize and trust region method.The new method uses the reciprocal of BB-stepsize to approximate the Hessian matrix of the objective function in the trust region subproblems,and accepts some bad solutions according to the modified Metropolis criterion based on simulated annealing idea.Under some suitable assumptions,the global convergence of the new method is established.Some preliminary numerical results indicate that,the new method is more efficient compared with the existing trust region BB method.
基金the support from the National Natural Science Foundation of China(Nos.12171384,12201492,61976176)the National Science Foundation of Shaanxi(No.2021JM-323).
文摘In this paper,we develop an active set identification technique.By means of the active set technique,we present an active set adaptive monotone projected Barzilai-Borwein method(ASAMPBB)for solving nonnegative matrix factorization(NMF)based on the alternating nonnegative least squares framework,in which the Barzilai-Borwein(BB)step sizes can be adaptively picked to get meaningful convergence rate improvements.To get optimal step size,we take into account of the curvature information.In addition,the larger step size technique is exploited to accelerate convergence of the proposed method.The global convergence of the proposed method is analysed under mild assumption.Finally,the results of the numerical experiments on both synthetic and real-world datasets show that the proposed method is effective.
