Smart grids are the developmental trend of power systems and they have attracted much attention all over the world.Due to their complexities,and the uncertainty of the smart grid and high volume of information being c...Smart grids are the developmental trend of power systems and they have attracted much attention all over the world.Due to their complexities,and the uncertainty of the smart grid and high volume of information being collected,artificial intelligence techniques represent some of the enabling technologies for its future development and success.Owing to the decreasing cost of computing power,the profusion of data,and better algorithms,AI has entered into its new develop-mental stage and AI 2.0 is developing rapidly.Deep learning(DL),reinforcement learning(RL)and their combination-deep reinforcement learning(DRL)are representative methods and relatively mature methods in the family of AI 2.0.This article introduces the concept and status quo of the above three methods,summarizes their potential for application in smart grids,and provides an overview of the research work on their application in smart grids.展开更多
With the growing integration of distributed energy resources(DERs),flexible loads,and other emerging technologies,there are increasing complexities and uncertainties for modern power and energy systems.This brings gre...With the growing integration of distributed energy resources(DERs),flexible loads,and other emerging technologies,there are increasing complexities and uncertainties for modern power and energy systems.This brings great challenges to the operation and control.Besides,with the deployment of advanced sensor and smart meters,a large number of data are generated,which brings opportunities for novel data-driven methods to deal with complicated operation and control issues.Among them,reinforcement learning(RL)is one of the most widely promoted methods for control and optimization problems.This paper provides a comprehensive literature review of RL in terms of basic ideas,various types of algorithms,and their applications in power and energy systems.The challenges and further works are also discussed.展开更多
Q-learning is a popular temporal-difference reinforcement learning algorithm which often explicitly stores state values using lookup tables. This implementation has been proven to converge to the optimal solution, but...Q-learning is a popular temporal-difference reinforcement learning algorithm which often explicitly stores state values using lookup tables. This implementation has been proven to converge to the optimal solution, but it is often beneficial to use a function-approximation system, such as deep neural networks, to estimate state values. It has been previously observed that Q-learning can be unstable when using value function approximation or when operating in a stochastic environment. This instability can adversely affect the algorithm’s ability to maximize its returns. In this paper, we present a new algorithm called Multi Q-learning to attempt to overcome the instability seen in Q-learning. We test our algorithm on a 4 × 4 grid-world with different stochastic reward functions using various deep neural networks and convolutional networks. Our results show that in most cases, Multi Q-learning outperforms Q-learning, achieving average returns up to 2.5 times higher than Q-learning and having a standard deviation of state values as low as 0.58.展开更多
In this paper,the deformation reinforcement theory(DRT) proposed by the authors is elaborated with a new definition of instability that an elasto-plastic structure is not stable if it cannot satisfy simultaneously equ...In this paper,the deformation reinforcement theory(DRT) proposed by the authors is elaborated with a new definition of instability that an elasto-plastic structure is not stable if it cannot satisfy simultaneously equilibrium condition,kinematical admissibility and constitutive equations under the prescribed loading.Starting from the definition,a proof is established to the principle of minimum plastic complementary energy for failured structures.It is revealed that the principle of mini-mum plastic complementary energy results in relaxed constitutive equations,especially,yield conditions.It is demonstrated with case studies that many key issues in arch dam design,e.g.,global stability,dam-toe reinforcement,dam-toe cracking,dam-abut-ment reinforcement,can be well solved within the framework of the deformation reinforcement theory.The structural global stability can be described by the curve of the plastic complementary energy vs overloading factor.The unbalanced-forces obtained by elasto-plastic FEM can be used as the basis of analysis of global stability,dam-heel cracking,dam-toe anchorage and reinforcement of faults of high arch dams and their foundations.展开更多
To assess the effectiveness of vacuum preloading combined electroosmotic strengthening of ultra-soft soil and study the mechanism of the process, a comprehensive experimental investigation was performed. A laboratory ...To assess the effectiveness of vacuum preloading combined electroosmotic strengthening of ultra-soft soil and study the mechanism of the process, a comprehensive experimental investigation was performed. A laboratory test cell was designed and applied to evaluate the vacuum preloading combined electroosmosis. Several factors were taken into consideration, including the directions of the electroosmotic current and water induced by vacuum preloading and the replenishment of groundwater from the surrounding area. The results indicate that electroosmosis together with vacuum preloading improve the soil strength greatly, with an increase of approximately 60%, and reduce the water content of the soil on the basis of consolidation of vacuum preloading, howeve~ further settlement is not obvious with only 1.7 mm. The reinforcement effect of vacuum preloading combined electroosmosis is better than that of electroosmosis after vacuum preloading. Elemental analysis using X-ray fluorescence proves that the soil strengthening during electroosmotic period in this work is mainly caused by electroosmosis-induced electrochemical reactions, the concentrations of Al2O3 in the VPCEO region increase by 2.2%, 1.5%, and 0.9% at the anode, the midpoint between the electrodes, and the cathode, respectively.展开更多
文摘Smart grids are the developmental trend of power systems and they have attracted much attention all over the world.Due to their complexities,and the uncertainty of the smart grid and high volume of information being collected,artificial intelligence techniques represent some of the enabling technologies for its future development and success.Owing to the decreasing cost of computing power,the profusion of data,and better algorithms,AI has entered into its new develop-mental stage and AI 2.0 is developing rapidly.Deep learning(DL),reinforcement learning(RL)and their combination-deep reinforcement learning(DRL)are representative methods and relatively mature methods in the family of AI 2.0.This article introduces the concept and status quo of the above three methods,summarizes their potential for application in smart grids,and provides an overview of the research work on their application in smart grids.
基金supported by the Sichuan Science and Technology Program(Sichuan Distinguished Young Scholars)(No.2020JDJQ0037).
文摘With the growing integration of distributed energy resources(DERs),flexible loads,and other emerging technologies,there are increasing complexities and uncertainties for modern power and energy systems.This brings great challenges to the operation and control.Besides,with the deployment of advanced sensor and smart meters,a large number of data are generated,which brings opportunities for novel data-driven methods to deal with complicated operation and control issues.Among them,reinforcement learning(RL)is one of the most widely promoted methods for control and optimization problems.This paper provides a comprehensive literature review of RL in terms of basic ideas,various types of algorithms,and their applications in power and energy systems.The challenges and further works are also discussed.
文摘Q-learning is a popular temporal-difference reinforcement learning algorithm which often explicitly stores state values using lookup tables. This implementation has been proven to converge to the optimal solution, but it is often beneficial to use a function-approximation system, such as deep neural networks, to estimate state values. It has been previously observed that Q-learning can be unstable when using value function approximation or when operating in a stochastic environment. This instability can adversely affect the algorithm’s ability to maximize its returns. In this paper, we present a new algorithm called Multi Q-learning to attempt to overcome the instability seen in Q-learning. We test our algorithm on a 4 × 4 grid-world with different stochastic reward functions using various deep neural networks and convolutional networks. Our results show that in most cases, Multi Q-learning outperforms Q-learning, achieving average returns up to 2.5 times higher than Q-learning and having a standard deviation of state values as low as 0.58.
基金Supported by the National Natural Science Foundation of China (Grant No.50709014)the Special Funds for Major State Basic Research Projects of China (Grant No.2009CB724604)
文摘In this paper,the deformation reinforcement theory(DRT) proposed by the authors is elaborated with a new definition of instability that an elasto-plastic structure is not stable if it cannot satisfy simultaneously equilibrium condition,kinematical admissibility and constitutive equations under the prescribed loading.Starting from the definition,a proof is established to the principle of minimum plastic complementary energy for failured structures.It is revealed that the principle of mini-mum plastic complementary energy results in relaxed constitutive equations,especially,yield conditions.It is demonstrated with case studies that many key issues in arch dam design,e.g.,global stability,dam-toe reinforcement,dam-toe cracking,dam-abut-ment reinforcement,can be well solved within the framework of the deformation reinforcement theory.The structural global stability can be described by the curve of the plastic complementary energy vs overloading factor.The unbalanced-forces obtained by elasto-plastic FEM can be used as the basis of analysis of global stability,dam-heel cracking,dam-toe anchorage and reinforcement of faults of high arch dams and their foundations.
基金Project(2009B13014) supported by the Fundamental Research Funds for the Central Universities of ChinaProject(IRT1125) supported by the Program for Changjiang Scholars and Innovative Research Team in University,China
文摘To assess the effectiveness of vacuum preloading combined electroosmotic strengthening of ultra-soft soil and study the mechanism of the process, a comprehensive experimental investigation was performed. A laboratory test cell was designed and applied to evaluate the vacuum preloading combined electroosmosis. Several factors were taken into consideration, including the directions of the electroosmotic current and water induced by vacuum preloading and the replenishment of groundwater from the surrounding area. The results indicate that electroosmosis together with vacuum preloading improve the soil strength greatly, with an increase of approximately 60%, and reduce the water content of the soil on the basis of consolidation of vacuum preloading, howeve~ further settlement is not obvious with only 1.7 mm. The reinforcement effect of vacuum preloading combined electroosmosis is better than that of electroosmosis after vacuum preloading. Elemental analysis using X-ray fluorescence proves that the soil strengthening during electroosmotic period in this work is mainly caused by electroosmosis-induced electrochemical reactions, the concentrations of Al2O3 in the VPCEO region increase by 2.2%, 1.5%, and 0.9% at the anode, the midpoint between the electrodes, and the cathode, respectively.
