Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and ...Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.展开更多
Because of limited resource of embedded platforms, the computational complexity of advanced control algorithms raises significant challenges for the use of embedded systems in complex control field. A Scilab/Scicos ba...Because of limited resource of embedded platforms, the computational complexity of advanced control algorithms raises significant challenges for the use of embedded systems in complex control field. A Scilab/Scicos based embedded controller is developed on which various control software can be easily modeled, simulated, implemented, and evaluated to meet the ever-expanding requirements of industrial control applications. Built on the Cirrus Logic EP9315 ARM systems-on-chip board, this embedded controller is possible to develop complex embedded control systems that employ advanced control strategies in a rapid and cost-efficient fashion. Due to the free and open source nature of the software packages used, the cost of the embedded controller is minimized.展开更多
Bacteria, like industrial engineers, must manage processes that convert low value inputs into high value outputs. Bacteria are not intelligent, so they utilize self-organizing production systems to accelerate life-sus...Bacteria, like industrial engineers, must manage processes that convert low value inputs into high value outputs. Bacteria are not intelligent, so they utilize self-organizing production systems to accelerate life-sustaining chemical processes. Here I explore two questions. First, can businesses apply the principles of self-organization? Second, can operations researchers contribute to our understanding of biological systems? I explain biochemical concepts in plain terms, illustrated with a few informative laboratory evolution experiments, and describe the organizing principles that underlie complex biological systems. I describe the new disciplines of synthetic biology and metabolic engineering, which offer opportunities for interdisciplinary collaboration between life scientists and operations researchers.展开更多
A complex autonomous inventory coupled system is considered. It can take, for example, the form of a network of chemical or biochemical reactors, where the inventory interactions perform the recycling of by-products b...A complex autonomous inventory coupled system is considered. It can take, for example, the form of a network of chemical or biochemical reactors, where the inventory interactions perform the recycling of by-products between the subsystems. Because of the flexible subsystems interactions, each of them can be operated with their own periods utilizing advantageously their dynamic properties. A multifrequency second-order test generalizing the p-test for single systems is described. It can be used to decide which kind of the operation (the static one, the periodic one or the multiperiodic one) will intensify the productivity of a complex system. An illustrative example of the multiperiodic optimization of a complex chemical production system is presented.展开更多
In this paper, an adaptive neuro-control structure for complex dynamic system is proposed. A recurrent Neural Network is trained-off-line to learn the inverse dynamics of the system from the observation of the input-o...In this paper, an adaptive neuro-control structure for complex dynamic system is proposed. A recurrent Neural Network is trained-off-line to learn the inverse dynamics of the system from the observation of the input-output data. The direct adaptive approach is performed after the training process is achieved. A lyapunov-Base training algorithm is proposed and used to adjust on-line the network weights so that the neural model output follows the desired one. The simulation results obtained verify the effectiveness of the proposed control method.展开更多
This paper, at the first time, considers the problem of decentralized variable structure control of complex giant singular uncertainty systems by using the property of diagonally dominant matrix and Frobenius-Person t...This paper, at the first time, considers the problem of decentralized variable structure control of complex giant singular uncertainty systems by using the property of diagonally dominant matrix and Frobenius-Person theorem. The splendid selection of switching manifold for each subsystem makes the design relatively straightforward and can be easily realized. An illustrate example is given.展开更多
The Journal of Control Theory and Applications （JCTA） publishes high-quality papers on the theory,design,and applications in the systems and control field.This journal is published quarterly.Three types of contribut...The Journal of Control Theory and Applications （JCTA） publishes high-quality papers on the theory,design,and applications in the systems and control field.This journal is published quarterly.Three types of contribution s are regularly considered：展开更多
基金supported in part by the National Natural Science Foundation of China(62222301, 62073085, 62073158, 61890930-5, 62021003)the National Key Research and Development Program of China (2021ZD0112302, 2021ZD0112301, 2018YFC1900800-5)Beijing Natural Science Foundation (JQ19013)。
文摘Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.
基金supported in part by the National Natural Science Foundation under Grant No.61070003,No.61272020,and No.61071128Zhejiang Provincial Natural Science Foundation under Grant No.R1090052 and No.Y1101184
文摘Because of limited resource of embedded platforms, the computational complexity of advanced control algorithms raises significant challenges for the use of embedded systems in complex control field. A Scilab/Scicos based embedded controller is developed on which various control software can be easily modeled, simulated, implemented, and evaluated to meet the ever-expanding requirements of industrial control applications. Built on the Cirrus Logic EP9315 ARM systems-on-chip board, this embedded controller is possible to develop complex embedded control systems that employ advanced control strategies in a rapid and cost-efficient fashion. Due to the free and open source nature of the software packages used, the cost of the embedded controller is minimized.
文摘Bacteria, like industrial engineers, must manage processes that convert low value inputs into high value outputs. Bacteria are not intelligent, so they utilize self-organizing production systems to accelerate life-sustaining chemical processes. Here I explore two questions. First, can businesses apply the principles of self-organization? Second, can operations researchers contribute to our understanding of biological systems? I explain biochemical concepts in plain terms, illustrated with a few informative laboratory evolution experiments, and describe the organizing principles that underlie complex biological systems. I describe the new disciplines of synthetic biology and metabolic engineering, which offer opportunities for interdisciplinary collaboration between life scientists and operations researchers.
文摘A complex autonomous inventory coupled system is considered. It can take, for example, the form of a network of chemical or biochemical reactors, where the inventory interactions perform the recycling of by-products between the subsystems. Because of the flexible subsystems interactions, each of them can be operated with their own periods utilizing advantageously their dynamic properties. A multifrequency second-order test generalizing the p-test for single systems is described. It can be used to decide which kind of the operation (the static one, the periodic one or the multiperiodic one) will intensify the productivity of a complex system. An illustrative example of the multiperiodic optimization of a complex chemical production system is presented.
文摘In this paper, an adaptive neuro-control structure for complex dynamic system is proposed. A recurrent Neural Network is trained-off-line to learn the inverse dynamics of the system from the observation of the input-output data. The direct adaptive approach is performed after the training process is achieved. A lyapunov-Base training algorithm is proposed and used to adjust on-line the network weights so that the neural model output follows the desired one. The simulation results obtained verify the effectiveness of the proposed control method.
文摘This paper, at the first time, considers the problem of decentralized variable structure control of complex giant singular uncertainty systems by using the property of diagonally dominant matrix and Frobenius-Person theorem. The splendid selection of switching manifold for each subsystem makes the design relatively straightforward and can be easily realized. An illustrate example is given.
文摘The Journal of Control Theory and Applications （JCTA） publishes high-quality papers on the theory,design,and applications in the systems and control field.This journal is published quarterly.Three types of contribution s are regularly considered：
