In this paper, based upon the basic solution of sink, the approximate solution of single drain hole in finite elements is derived by use of the superposition principle. Then, the theoretical solution is extended to th...In this paper, based upon the basic solution of sink, the approximate solution of single drain hole in finite elements is derived by use of the superposition principle. Then, the theoretical solution is extended to the case of some drain holes in one finite element, and the method is used in seepage control analysis with quick convergence and high accuracy. On the other hand, if the positions of the drain holes are changed, only some control factors of drain holes are changed, but the finite element grid need not to be reformed. Therefore, the method is more suitable in optimal research of seepage control.展开更多
The core task of tracking control is to make the controlled plant track a desired trajectory.The traditional performance index used in previous studies cannot eliminate completely the tracking error as the number of t...The core task of tracking control is to make the controlled plant track a desired trajectory.The traditional performance index used in previous studies cannot eliminate completely the tracking error as the number of time steps increases.In this paper,a new cost function is introduced to develop the value-iteration-based adaptive critic framework to solve the tracking control problem.Unlike the regulator problem,the iterative value function of tracking control problem cannot be regarded as a Lyapunov function.A novel stability analysis method is developed to guarantee that the tracking error converges to zero.The discounted iterative scheme under the new cost function for the special case of linear systems is elaborated.Finally,the tracking performance of the present scheme is demonstrated by numerical results and compared with those of the traditional approaches.展开更多
In this paper, a novel iterative Q-learning algorithm, called "policy iteration based deterministic Qlearning algorithm", is developed to solve the optimal control problems for discrete-time deterministic no...In this paper, a novel iterative Q-learning algorithm, called "policy iteration based deterministic Qlearning algorithm", is developed to solve the optimal control problems for discrete-time deterministic nonlinear systems. The idea is to use an iterative adaptive dynamic programming(ADP) technique to construct the iterative control law which optimizes the iterative Q function. When the optimal Q function is obtained, the optimal control law can be achieved by directly minimizing the optimal Q function, where the mathematical model of the system is not necessary. Convergence property is analyzed to show that the iterative Q function is monotonically non-increasing and converges to the solution of the optimality equation. It is also proven that any of the iterative control laws is a stable control law. Neural networks are employed to implement the policy iteration based deterministic Q-learning algorithm, by approximating the iterative Q function and the iterative control law, respectively. Finally, two simulation examples are presented to illustrate the performance of the developed algorithm.展开更多
Solving optimal control problems serves as the basic demand of industrial control tasks.Existing methods like model predictive control often suffer from heavy online computational burdens.Reinforcement learning has sh...Solving optimal control problems serves as the basic demand of industrial control tasks.Existing methods like model predictive control often suffer from heavy online computational burdens.Reinforcement learning has shown promise in computer and board games but has yet to be widely adopted in industrial applications due to a lack of accessible,high-accuracy solvers.Current Reinforcement learning(RL)solvers are often developed for academic research and require a significant amount of theoretical knowledge and programming skills.Besides,many of them only support Python-based environments and limit to model-free algorithms.To address this gap,this paper develops General Optimal control Problems Solver(GOPS),an easy-to-use RL solver package that aims to build real-time and high-performance controllers in industrial fields.GOPS is built with a highly modular structure that retains a flexible framework for secondary development.Considering the diversity of industrial control tasks,GOPS also includes a conversion tool that allows for the use of Matlab/Simulink to support environment construction,controller design,and performance validation.To handle large-scale problems,GOPS can automatically create various serial and parallel trainers by flexibly combining embedded buffers and samplers.It offers a variety of common approximate functions for policy and value functions,including polynomial,multilayer perceptron,convolutional neural network,etc.Additionally,constrained and robust algorithms for special industrial control systems with state constraints and model uncertainties are also integrated into GOPS.Several examples,including linear quadratic control,inverted double pendulum,vehicle tracking,humanoid robot,obstacle avoidance,and active suspension control,are tested to verify the performances of GOPS.展开更多
In this paper,the approximate synchronization of leader-follower multiagent systems(MASs) over finite fields is studied in regard to local and global synchronization.First,the approximately synchronous state set(ASSS)...In this paper,the approximate synchronization of leader-follower multiagent systems(MASs) over finite fields is studied in regard to local and global synchronization.First,the approximately synchronous state set(ASSS) is obtained.Second,combined with ASSS and transient periods,some criteria for the local and global approximate synchronization of systems are given.Moreover,the algorithms for calculating the maximum approximately synchronous basin(MASB) and the maximum control invariant set(MCIS) are presented.Third,the global approximate synchronization of the system is achieved by designing the state feedback control,and a design algorithm of the controller using the truth matrix method is proposed.Moreover,the results of approximate synchronization are degenerated to complete synchronization.Last,two examples are shown to demonstrate the results of this paper.展开更多
The article proposes a nonlinear optimal(H-infinity)control approach for the model of a tracked robotic vehicle.The kinematic model of such a tracked vehicle takes into account slippage effects due to the contact of t...The article proposes a nonlinear optimal(H-infinity)control approach for the model of a tracked robotic vehicle.The kinematic model of such a tracked vehicle takes into account slippage effects due to the contact of the tracks with the ground.To solve the related control problem,the dynamic model of the vehicle undergoes first approximate linearization around a temporary operating point which is updated at each iteration of the control algorithm.The linearization process relies on first-order Taylor series expansion and on the computation of the Jacobian matrices of the state-space model of the vehicle.For the approximately linearized description of the tracked vehicle a stabilizing H-infinity feedback controller is designed.To compute the controller’s feedback gains an algebraic Riccati equation is solved at each time-step of the control method.The stability properties of the control scheme are proven through Lyapunov analysis.It is also demonstrated that the control method retains the advantages of linear optimal control,that is fast and accurate tracking of reference setpoints under moderate variations of the control inputs.展开更多
This paper proposes a novel excitation controller using support vector machines （SVM） and approximate models. The nonlinear control law is derived directly based on an input-output approximation method via Taylor ex...This paper proposes a novel excitation controller using support vector machines （SVM） and approximate models. The nonlinear control law is derived directly based on an input-output approximation method via Taylor expansion, which not only avoids complex control development and intensive computation, but also avoids online learning or adjustment. Only a general SVM modelling technique is involved in both model identification and controller implementation. The robustness of the stability is rigorously established using the Lyapunov method. Several simulations demonstrate the effectiveness of the proposed excitation controller.展开更多
Profile requirements of silicon steel strip are extremely high and the thickness difference of cold-rolled products is usually less than 7μm,and the profile quality of hot-rolled strip is the key to ensure the thickn...Profile requirements of silicon steel strip are extremely high and the thickness difference of cold-rolled products is usually less than 7μm,and the profile quality of hot-rolled strip is the key to ensure the thickness difference of cold-rolled products.In order to produce the silicon steel strip with high-precision shape,the concept of quasi-rectangular rolling during hot continuous rolling was put forward;the equipment configuration and technical method of approximate rectangular section control were studied.Through the roughing multi-target load distribution technology and the roll configuration technology for uniform wear of a 4-high rolling mill,the strip crown of transfer bar was reduced and the profile control stability was guaranteed.Configuring variable contact back-up roll technology on all stands in the finishing rolling process,equipped with symmetry variable taper work roll and long-stroke intelligent shifting strategy in the downstream stands,and using side rolling lubrication technology can make the roll wear more uniform,reduce the edge drop of silicon steel strip,improve the profile quality,and make the strip section of finishing exit"quasi-rectangular".In addition,induction furnace and side heater were also equipped to guarantee the temperature uniformity of the strip,so as to improve the stability of profile control.The whole control technology is based on the 1580-mm hot continuous rolling production line,designed,and developed according to the characteristics of equipment and products,and has been successfully applied,which can obtain the approximate rectangular strip section satisfying the flatness quality,and improve the strip section precision of silicon steel and other products.展开更多
Aimed at infinite horizon optimal control problems of discrete time-varying nonlinear systems,in this paper,a new iterative adaptive dynamic programming algorithm,which is the discrete-time time-varying policy iterati...Aimed at infinite horizon optimal control problems of discrete time-varying nonlinear systems,in this paper,a new iterative adaptive dynamic programming algorithm,which is the discrete-time time-varying policy iteration(DTTV)algorithm,is developed.The iterative control law is designed to update the iterative value function which approximates the index function of optimal performance.The admissibility of the iterative control law is analyzed.The results show that the iterative value function is non-increasingly convergent to the Bellman-equation optimal solution.To implement the algorithm,neural networks are employed and a new implementation structure is established,which avoids solving the generalized Bellman equation in each iteration.Finally,the optimal control laws for torsional pendulum and inverted pendulum systems are obtained by using the DTTV policy iteration algorithm,where the mass and pendulum bar length are permitted to be time-varying parameters.The effectiveness of the developed method is illustrated by numerical results and comparisons.展开更多
文摘In this paper, based upon the basic solution of sink, the approximate solution of single drain hole in finite elements is derived by use of the superposition principle. Then, the theoretical solution is extended to the case of some drain holes in one finite element, and the method is used in seepage control analysis with quick convergence and high accuracy. On the other hand, if the positions of the drain holes are changed, only some control factors of drain holes are changed, but the finite element grid need not to be reformed. Therefore, the method is more suitable in optimal research of seepage control.
基金This work was supported in part by Beijing Natural Science Foundation(JQ19013)the National Key Research and Development Program of China(2021ZD0112302)the National Natural Science Foundation of China(61773373).
文摘The core task of tracking control is to make the controlled plant track a desired trajectory.The traditional performance index used in previous studies cannot eliminate completely the tracking error as the number of time steps increases.In this paper,a new cost function is introduced to develop the value-iteration-based adaptive critic framework to solve the tracking control problem.Unlike the regulator problem,the iterative value function of tracking control problem cannot be regarded as a Lyapunov function.A novel stability analysis method is developed to guarantee that the tracking error converges to zero.The discounted iterative scheme under the new cost function for the special case of linear systems is elaborated.Finally,the tracking performance of the present scheme is demonstrated by numerical results and compared with those of the traditional approaches.
基金supported in part by National Natural Science Foundation of China(Grant Nos.6137410561233001+1 种基金61273140)in part by Beijing Natural Science Foundation(Grant No.4132078)
文摘In this paper, a novel iterative Q-learning algorithm, called "policy iteration based deterministic Qlearning algorithm", is developed to solve the optimal control problems for discrete-time deterministic nonlinear systems. The idea is to use an iterative adaptive dynamic programming(ADP) technique to construct the iterative control law which optimizes the iterative Q function. When the optimal Q function is obtained, the optimal control law can be achieved by directly minimizing the optimal Q function, where the mathematical model of the system is not necessary. Convergence property is analyzed to show that the iterative Q function is monotonically non-increasing and converges to the solution of the optimality equation. It is also proven that any of the iterative control laws is a stable control law. Neural networks are employed to implement the policy iteration based deterministic Q-learning algorithm, by approximating the iterative Q function and the iterative control law, respectively. Finally, two simulation examples are presented to illustrate the performance of the developed algorithm.
基金supported by the National Key R&D Program of China(2022YFB2502901)the Natural Science Foundation of China(U20A20334).
文摘Solving optimal control problems serves as the basic demand of industrial control tasks.Existing methods like model predictive control often suffer from heavy online computational burdens.Reinforcement learning has shown promise in computer and board games but has yet to be widely adopted in industrial applications due to a lack of accessible,high-accuracy solvers.Current Reinforcement learning(RL)solvers are often developed for academic research and require a significant amount of theoretical knowledge and programming skills.Besides,many of them only support Python-based environments and limit to model-free algorithms.To address this gap,this paper develops General Optimal control Problems Solver(GOPS),an easy-to-use RL solver package that aims to build real-time and high-performance controllers in industrial fields.GOPS is built with a highly modular structure that retains a flexible framework for secondary development.Considering the diversity of industrial control tasks,GOPS also includes a conversion tool that allows for the use of Matlab/Simulink to support environment construction,controller design,and performance validation.To handle large-scale problems,GOPS can automatically create various serial and parallel trainers by flexibly combining embedded buffers and samplers.It offers a variety of common approximate functions for policy and value functions,including polynomial,multilayer perceptron,convolutional neural network,etc.Additionally,constrained and robust algorithms for special industrial control systems with state constraints and model uncertainties are also integrated into GOPS.Several examples,including linear quadratic control,inverted double pendulum,vehicle tracking,humanoid robot,obstacle avoidance,and active suspension control,are tested to verify the performances of GOPS.
基金supported by the National Natural Science Foundation of China under Grant Nos.62373178,62273201,and 62103176the Research Fundfor the Taishan Scholar Project of Shandong Province of China under Grant Nos.tstp20221103 and tstp20221103。
文摘In this paper,the approximate synchronization of leader-follower multiagent systems(MASs) over finite fields is studied in regard to local and global synchronization.First,the approximately synchronous state set(ASSS) is obtained.Second,combined with ASSS and transient periods,some criteria for the local and global approximate synchronization of systems are given.Moreover,the algorithms for calculating the maximum approximately synchronous basin(MASB) and the maximum control invariant set(MCIS) are presented.Third,the global approximate synchronization of the system is achieved by designing the state feedback control,and a design algorithm of the controller using the truth matrix method is proposed.Moreover,the results of approximate synchronization are degenerated to complete synchronization.Last,two examples are shown to demonstrate the results of this paper.
基金supported by the Research“Advances in Applied Nonlinear Optimal Control”under Grant No.6065。
文摘The article proposes a nonlinear optimal(H-infinity)control approach for the model of a tracked robotic vehicle.The kinematic model of such a tracked vehicle takes into account slippage effects due to the contact of the tracks with the ground.To solve the related control problem,the dynamic model of the vehicle undergoes first approximate linearization around a temporary operating point which is updated at each iteration of the control algorithm.The linearization process relies on first-order Taylor series expansion and on the computation of the Jacobian matrices of the state-space model of the vehicle.For the approximately linearized description of the tracked vehicle a stabilizing H-infinity feedback controller is designed.To compute the controller’s feedback gains an algebraic Riccati equation is solved at each time-step of the control method.The stability properties of the control scheme are proven through Lyapunov analysis.It is also demonstrated that the control method retains the advantages of linear optimal control,that is fast and accurate tracking of reference setpoints under moderate variations of the control inputs.
基金the National Natural Science Foundation of China (No.60375001,60775047,60402024).
文摘This paper proposes a novel excitation controller using support vector machines （SVM） and approximate models. The nonlinear control law is derived directly based on an input-output approximation method via Taylor expansion, which not only avoids complex control development and intensive computation, but also avoids online learning or adjustment. Only a general SVM modelling technique is involved in both model identification and controller implementation. The robustness of the stability is rigorously established using the Lyapunov method. Several simulations demonstrate the effectiveness of the proposed excitation controller.
基金This work was financially supported by the National Natural Science Foundation of China(Grant No.51975043)Fundamental Research Funds for the Central Universities(Grant No.FRF-TP-19-002A3)Beijing Natural Science Foundation(3182026)for the support to this research.
文摘Profile requirements of silicon steel strip are extremely high and the thickness difference of cold-rolled products is usually less than 7μm,and the profile quality of hot-rolled strip is the key to ensure the thickness difference of cold-rolled products.In order to produce the silicon steel strip with high-precision shape,the concept of quasi-rectangular rolling during hot continuous rolling was put forward;the equipment configuration and technical method of approximate rectangular section control were studied.Through the roughing multi-target load distribution technology and the roll configuration technology for uniform wear of a 4-high rolling mill,the strip crown of transfer bar was reduced and the profile control stability was guaranteed.Configuring variable contact back-up roll technology on all stands in the finishing rolling process,equipped with symmetry variable taper work roll and long-stroke intelligent shifting strategy in the downstream stands,and using side rolling lubrication technology can make the roll wear more uniform,reduce the edge drop of silicon steel strip,improve the profile quality,and make the strip section of finishing exit"quasi-rectangular".In addition,induction furnace and side heater were also equipped to guarantee the temperature uniformity of the strip,so as to improve the stability of profile control.The whole control technology is based on the 1580-mm hot continuous rolling production line,designed,and developed according to the characteristics of equipment and products,and has been successfully applied,which can obtain the approximate rectangular strip section satisfying the flatness quality,and improve the strip section precision of silicon steel and other products.
基金supported in part by Fundamental Research Funds for the Central Universities(2022JBZX024)in part by the National Natural Science Foundation of China(61872037,61273167)。
文摘Aimed at infinite horizon optimal control problems of discrete time-varying nonlinear systems,in this paper,a new iterative adaptive dynamic programming algorithm,which is the discrete-time time-varying policy iteration(DTTV)algorithm,is developed.The iterative control law is designed to update the iterative value function which approximates the index function of optimal performance.The admissibility of the iterative control law is analyzed.The results show that the iterative value function is non-increasingly convergent to the Bellman-equation optimal solution.To implement the algorithm,neural networks are employed and a new implementation structure is established,which avoids solving the generalized Bellman equation in each iteration.Finally,the optimal control laws for torsional pendulum and inverted pendulum systems are obtained by using the DTTV policy iteration algorithm,where the mass and pendulum bar length are permitted to be time-varying parameters.The effectiveness of the developed method is illustrated by numerical results and comparisons.
