The relationship between the technique by state- dependent Riccati equations （SDRE） and Hamilton-Jacobi-lsaacs （HJI） equations for nonlinear H∞ control design is investigated. By establishing the Lyapunov matrix ...The relationship between the technique by state- dependent Riccati equations （SDRE） and Hamilton-Jacobi-lsaacs （HJI） equations for nonlinear H∞ control design is investigated. By establishing the Lyapunov matrix equations for partial derivates of the solution of the SDREs and introducing symmetry measure for some related matrices, a method is proposed for examining whether the SDRE method admits a global optimal control equiva- lent to that solved by the HJI equation method. Two examples with simulation are given to illustrate the method is effective.展开更多
In this paper, neural networks are used to approximately solve the finite-horizon constrained input H-infinity state feedback control problem. The method is based on solving a related Hamilton-Jacobi-Isaacs equation o...In this paper, neural networks are used to approximately solve the finite-horizon constrained input H-infinity state feedback control problem. The method is based on solving a related Hamilton-Jacobi-Isaacs equation of the corresponding finite-horizon zero-sum game. The game value function is approximated by a neural network with time- varying weights. It is shown that the neural network approximation converges uniformly to the game-value function and the resulting almost optimal constrained feedback controller provides closed-loop stability and bounded L2 gain. The result is an almost optimal H-infinity feedback controller with time-varying coefficients that is solved a priori off-line. The effectiveness of the method is shown on the Rotational/Translational Actuator benchmark nonlinear control problem.展开更多
This paper deals with the output feedback H∞ control problem for a class of nonlinear stochastic systems. Based on the latest developed theory of stochastic dissipation, a notable result about the nonlinear H∞ outpu...This paper deals with the output feedback H∞ control problem for a class of nonlinear stochastic systems. Based on the latest developed theory of stochastic dissipation, a notable result about the nonlinear H∞ output feedback control of deterministic system is generalized to the stochastic case. Finally, in the cases of state feedback and output feedback, two families of controllers are provided respectively.展开更多
Purpose–The air-breathing hypersonic vehicle(AHV)includes intricate inherent coupling between the propulsion system and the airframe dynamics,which results in an intractable nonlinear system for the controller design...Purpose–The air-breathing hypersonic vehicle(AHV)includes intricate inherent coupling between the propulsion system and the airframe dynamics,which results in an intractable nonlinear system for the controller design.The purpose of this paper is to propose an H1 control method for AHV based on the online simultaneous policy update algorithm(SPUA).Design/methodology/approach–Initially,the H1 state feedback control problem of the AHV is converted to the problem of solving the Hamilton-Jacobi-Isaacs(HJI)equation,which is notoriously difficult to solve both numerically and analytically.To overcome this difficulty,the online SPUA is introduced to solve the HJI equation without requiring the accurate knowledge of the internal system dynamics.Subsequently,the online SPUA is implemented on the basis of an actor-critic structure,in which neural network(NN)is employed for approximating the cost function and a least-square method is used to calculate the NN weight parameters.Findings–Simulation study on the AHV demonstrates the effectiveness of the proposed H1 control method.Originality/value–The paper presents an interesting method for the H1 state feedback control design problem of the AHV based on online SPUA.展开更多
Qualitative spacecraft pursuit-evasion problem which focuses on feasibility is rarely studied because of high-dimensional dynamics,intractable terminal constraints and heavy computational cost.In this paper,A physics-...Qualitative spacecraft pursuit-evasion problem which focuses on feasibility is rarely studied because of high-dimensional dynamics,intractable terminal constraints and heavy computational cost.In this paper,A physics-informed framework is proposed for the problem,providing an intuitive method for spacecraft threat relationship determination,situation assessment,mission feasibility analysis and orbital game rules summarization.For the first time,situation adjustment suggestions can be provided for the weak player in orbital game.First,a dimension-reduction dynamics is derived in the line-of-sight rotation coordinate system and the qualitative model is determined,reducing complexity and avoiding the difficulty of target set presentation caused by individual modeling.Second,the Backwards Reachable Set(BRS)of the target set is used for state space partition and capture zone presentation.Reverse-time analysis can eliminate the influence of changeable initial state and enable the proposed framework to analyze plural situations simultaneously.Third,a time-dependent Hamilton-Jacobi-Isaacs(HJI)Partial Differential Equation(PDE)is established to describe BRS evolution driven by dimension-reduction dynamics,based on level set method.Then,Physics-Informed Neural Networks(PINNs)are extended to HJI PDE final value problem,supporting orbital game rules summarization through capture zone evolution analysis.Finally,numerical results demonstrate the feasibility and efficiency of the proposed framework.展开更多
Missile interception problem can be regarded as a two-person zero-sum differential games problem,which depends on the solution of Hamilton-Jacobi-Isaacs(HJI)equa-tion.It has been proved impossible to obtain a closed-f...Missile interception problem can be regarded as a two-person zero-sum differential games problem,which depends on the solution of Hamilton-Jacobi-Isaacs(HJI)equa-tion.It has been proved impossible to obtain a closed-form solu-tion due to the nonlinearity of HJI equation,and many iterative algorithms are proposed to solve the HJI equation.Simultane-ous policy updating algorithm(SPUA)is an effective algorithm for solving HJI equation,but it is an on-policy integral reinforce-ment learning(IRL).For online implementation of SPUA,the dis-turbance signals need to be adjustable,which is unrealistic.In this paper,an off-policy IRL algorithm based on SPUA is pro-posed without making use of any knowledge of the systems dynamics.Then,a neural-network based online adaptive critic implementation scheme of the off-policy IRL algorithm is pre-sented.Based on the online off-policy IRL method,a computa-tional intelligence interception guidance(CIIG)law is developed for intercepting high-maneuvering target.As a model-free method,intercepting targets can be achieved through measur-ing system data online.The effectiveness of the CIIG is verified through two missile and target engagement scenarios.展开更多
In this paper, we propose an iterative relaxation method for solving the Hamilton-Jacobi-Bellman-Isaacs equation(HJBIE) arising in deterministic optimal control of affine nonlinear systems. Local convergence of the me...In this paper, we propose an iterative relaxation method for solving the Hamilton-Jacobi-Bellman-Isaacs equation(HJBIE) arising in deterministic optimal control of affine nonlinear systems. Local convergence of the method is established under fairly mild assumptions, and examples are solved to demonstrate the effectiveness of the method. An extension of the approach to Lyapunov equations is also discussed. The preliminary results presented are promising, and it is hoped that the approach will ultimately develop into an efficient computational tool for solving the HJBIEs.展开更多
This paper focuses on zero-sum stochastic differential games in the framework of forwardbackward stochastic differential equations on a finite time horizon with both players adopting impulse controls.By means of BSDE ...This paper focuses on zero-sum stochastic differential games in the framework of forwardbackward stochastic differential equations on a finite time horizon with both players adopting impulse controls.By means of BSDE methods,in particular that of the notion from Peng’s stochastic backward semigroups,the authors prove a dynamic programming principle for both the upper and the lower value functions of the game.The upper and the lower value functions are then shown to be the unique viscosity solutions of the Hamilton-Jacobi-Bellman-Isaacs equations with a double-obstacle.As a consequence,the uniqueness implies that the upper and lower value functions coincide and the game admits a value.展开更多
In this paper,the authors analyze the optimal reinsurance and dividend problem with model uncertainty for an insurer.Here the model uncertainty represents possible deviations between the real market and the assumed mo...In this paper,the authors analyze the optimal reinsurance and dividend problem with model uncertainty for an insurer.Here the model uncertainty represents possible deviations between the real market and the assumed model.In addition to the incorporation of model uncertainty into the traditional diffusion surplus process,the authors include a penalty function in the objective function.The proposed goal is to find the optimal reinsurance and dividend strategy that maximizes the expected discounted dividend before ruin in the worst case of all possible scenarios,namely,the worst market.Using a dynamic programming approach,the problem is reduced to solving a Hamilton-Jacob-Bellman-Isaac(HJBI)equation with singular control.This problem is more difficult than the traditional robust control or singular control problem.Here,the authors prove that the value function is the unique solution to this HJBI equation with singular control.Moreover,the authors present a verification theorem when a smooth solution can be found,and derive closed-form solution when the function in the objective function is specified.展开更多
基金supported by the National Natural Science Foundation of China(60874114)
文摘The relationship between the technique by state- dependent Riccati equations （SDRE） and Hamilton-Jacobi-lsaacs （HJI） equations for nonlinear H∞ control design is investigated. By establishing the Lyapunov matrix equations for partial derivates of the solution of the SDREs and introducing symmetry measure for some related matrices, a method is proposed for examining whether the SDRE method admits a global optimal control equiva- lent to that solved by the HJI equation method. Two examples with simulation are given to illustrate the method is effective.
基金This work was supported by the National Science Foundation (ECS-0501451)Army Research Office (W91NF-05-1-0314).
文摘In this paper, neural networks are used to approximately solve the finite-horizon constrained input H-infinity state feedback control problem. The method is based on solving a related Hamilton-Jacobi-Isaacs equation of the corresponding finite-horizon zero-sum game. The game value function is approximated by a neural network with time- varying weights. It is shown that the neural network approximation converges uniformly to the game-value function and the resulting almost optimal constrained feedback controller provides closed-loop stability and bounded L2 gain. The result is an almost optimal H-infinity feedback controller with time-varying coefficients that is solved a priori off-line. The effectiveness of the method is shown on the Rotational/Translational Actuator benchmark nonlinear control problem.
基金Supported by the National Natural Science Foundation of China (No. 60774020,60736028 and 60821091)
文摘This paper deals with the output feedback H∞ control problem for a class of nonlinear stochastic systems. Based on the latest developed theory of stochastic dissipation, a notable result about the nonlinear H∞ output feedback control of deterministic system is generalized to the stochastic case. Finally, in the cases of state feedback and output feedback, two families of controllers are provided respectively.
基金supported by the National Basic Research Program of China(973 Program)(2012CB720003)the National Natural Science Foundation of China under Grants 91016004,61074057 and 61121003.
文摘Purpose–The air-breathing hypersonic vehicle(AHV)includes intricate inherent coupling between the propulsion system and the airframe dynamics,which results in an intractable nonlinear system for the controller design.The purpose of this paper is to propose an H1 control method for AHV based on the online simultaneous policy update algorithm(SPUA).Design/methodology/approach–Initially,the H1 state feedback control problem of the AHV is converted to the problem of solving the Hamilton-Jacobi-Isaacs(HJI)equation,which is notoriously difficult to solve both numerically and analytically.To overcome this difficulty,the online SPUA is introduced to solve the HJI equation without requiring the accurate knowledge of the internal system dynamics.Subsequently,the online SPUA is implemented on the basis of an actor-critic structure,in which neural network(NN)is employed for approximating the cost function and a least-square method is used to calculate the NN weight parameters.Findings–Simulation study on the AHV demonstrates the effectiveness of the proposed H1 control method.Originality/value–The paper presents an interesting method for the H1 state feedback control design problem of the AHV based on online SPUA.
基金This study was supported by the Independent Innovation Science Foundation Project of National University of Defense Technology,China(No.22-ZZCX-083).
文摘Qualitative spacecraft pursuit-evasion problem which focuses on feasibility is rarely studied because of high-dimensional dynamics,intractable terminal constraints and heavy computational cost.In this paper,A physics-informed framework is proposed for the problem,providing an intuitive method for spacecraft threat relationship determination,situation assessment,mission feasibility analysis and orbital game rules summarization.For the first time,situation adjustment suggestions can be provided for the weak player in orbital game.First,a dimension-reduction dynamics is derived in the line-of-sight rotation coordinate system and the qualitative model is determined,reducing complexity and avoiding the difficulty of target set presentation caused by individual modeling.Second,the Backwards Reachable Set(BRS)of the target set is used for state space partition and capture zone presentation.Reverse-time analysis can eliminate the influence of changeable initial state and enable the proposed framework to analyze plural situations simultaneously.Third,a time-dependent Hamilton-Jacobi-Isaacs(HJI)Partial Differential Equation(PDE)is established to describe BRS evolution driven by dimension-reduction dynamics,based on level set method.Then,Physics-Informed Neural Networks(PINNs)are extended to HJI PDE final value problem,supporting orbital game rules summarization through capture zone evolution analysis.Finally,numerical results demonstrate the feasibility and efficiency of the proposed framework.
文摘Missile interception problem can be regarded as a two-person zero-sum differential games problem,which depends on the solution of Hamilton-Jacobi-Isaacs(HJI)equa-tion.It has been proved impossible to obtain a closed-form solu-tion due to the nonlinearity of HJI equation,and many iterative algorithms are proposed to solve the HJI equation.Simultane-ous policy updating algorithm(SPUA)is an effective algorithm for solving HJI equation,but it is an on-policy integral reinforce-ment learning(IRL).For online implementation of SPUA,the dis-turbance signals need to be adjustable,which is unrealistic.In this paper,an off-policy IRL algorithm based on SPUA is pro-posed without making use of any knowledge of the systems dynamics.Then,a neural-network based online adaptive critic implementation scheme of the off-policy IRL algorithm is pre-sented.Based on the online off-policy IRL method,a computa-tional intelligence interception guidance(CIIG)law is developed for intercepting high-maneuvering target.As a model-free method,intercepting targets can be achieved through measur-ing system data online.The effectiveness of the CIIG is verified through two missile and target engagement scenarios.
文摘In this paper, we propose an iterative relaxation method for solving the Hamilton-Jacobi-Bellman-Isaacs equation(HJBIE) arising in deterministic optimal control of affine nonlinear systems. Local convergence of the method is established under fairly mild assumptions, and examples are solved to demonstrate the effectiveness of the method. An extension of the approach to Lyapunov equations is also discussed. The preliminary results presented are promising, and it is hoped that the approach will ultimately develop into an efficient computational tool for solving the HJBIEs.
基金supported by the National Nature Science Foundation of China under Grant Nos.11701040,11871010,61871058the Fundamental Research Funds for the Central Universities under Grant No.2019XDA11。
文摘This paper focuses on zero-sum stochastic differential games in the framework of forwardbackward stochastic differential equations on a finite time horizon with both players adopting impulse controls.By means of BSDE methods,in particular that of the notion from Peng’s stochastic backward semigroups,the authors prove a dynamic programming principle for both the upper and the lower value functions of the game.The upper and the lower value functions are then shown to be the unique viscosity solutions of the Hamilton-Jacobi-Bellman-Isaacs equations with a double-obstacle.As a consequence,the uniqueness implies that the upper and lower value functions coincide and the game admits a value.
基金supported by the National Natural Science Foundation of China under Grant No. 11771466Program for Innovation Research under Grant No. 20170074the Emerging Interdisciplinary Project of CUFE
文摘In this paper,the authors analyze the optimal reinsurance and dividend problem with model uncertainty for an insurer.Here the model uncertainty represents possible deviations between the real market and the assumed model.In addition to the incorporation of model uncertainty into the traditional diffusion surplus process,the authors include a penalty function in the objective function.The proposed goal is to find the optimal reinsurance and dividend strategy that maximizes the expected discounted dividend before ruin in the worst case of all possible scenarios,namely,the worst market.Using a dynamic programming approach,the problem is reduced to solving a Hamilton-Jacob-Bellman-Isaac(HJBI)equation with singular control.This problem is more difficult than the traditional robust control or singular control problem.Here,the authors prove that the value function is the unique solution to this HJBI equation with singular control.Moreover,the authors present a verification theorem when a smooth solution can be found,and derive closed-form solution when the function in the objective function is specified.
