摘要
“双碳”目标加速了大规模新能源并网的新型电力系统的发展。传统控制方法无法有效解决分布式电网模式下新能源规模化接入所带来的强随机扰动,从而造成频率不稳定、控制性能标准(control performance standards,CPS)越来越差的问题。为此,该文从二次调频的角度提出一种能够保证具有较高的Q值更新学习率,且无论强随机环境亦或平稳环境均具有更稳定的响应特性的拉格朗日松弛强化学习算法,即重复更新Q学习拉格朗日松弛(repeated update Q-Learning using Lagrangian relaxation,RUQL-LR)算法,来获取多区域协同。对改进的IEEE标准两区域模型和以西南电网为基础的三区域模型进行仿真,验证了所提算法的有效性。该算法不仅能够在很大程度上提高Q值估计准确性,还能使弱耦合动态优化问题分散为多个子问题,以快速获取最优策略,且与多种强化学习算法相比,其Q值估计误差更小,能明显提高电网的频率稳定性。
The targets of carbon peak and carbon neutrality have accelerated the development of new power systems integrated large-scale new energy. In the distributed grid mode,large-scale access of new energy brings the strong random disturbance, which cannot be effectively solved by the traditional control methods, so that the frequency is unstable and the Control Performance Standards(CPS) is increasingly poor. Therefore, from the perspective of secondary frequency regulation, this paper proposes a Repeat Update Q-Learning using Lagrangian Relaxation algorithm(the so-called RUQL-LR algorithm) to obtain multi-area synergy. On the one hand, it can guarantee a higher Q value update learning rate.On the other hand, it has more stable response characteristics in both strong random environment and stable environment. In order to verify the effectiveness of the proposed algorithm, the improved IEEE standard two-area model and the three-area model based on Southwest Power Grid are simulated. The results show that the proposed algorithm can improve the accuracy of Q value estimation to a large extent. In addition, it disperses the weak coupling dynamic optimization problem into multiple sub-problems, so that it can quickly obtain the optimal strategy. Compared with multiple reinforcement learning algorithms, its Q-value estimation bias is smaller which could significantly improve the frequency stability of power grid.
作者
席磊
刘治洪
李彦营
XI Lei;LIU Zhihong;LI Yanying(College of Electrical Engineering and New Energy,China Three Gorges University,Yichang 443002,Hubei Province,China)
出处
《中国电机工程学报》
EI
CSCD
北大核心
2023年第4期1359-1368,共10页
Proceedings of the CSEE
基金
国家自然科学基金项目(52277108)。
关键词
自动发电控制
多智能体
强化学习
拉格朗日松弛
automatic generation control
multi-agent
reinforcement learning
Lagrangian relaxation
