摘要
演化计算利用生物演化过程中的自然选择机制和遗传规律求解优化问题,循环神经网络的精度和效率依赖其参数以及结构的优化效果,采用演化计算解决循环神经网络中的参数与结构自适应优化问题是自动化深度学习领域的研究热点。文中针对结合演化计算和循环神经网络的算法进行了详细的调研。首先,简要介绍了演化算法的传统类别、常见算法和优点,以及循环神经网络模型的结构及特点,并对影响循环神经网络性能的因素进行了分析;其次,分析了演化循环神经网络的算法框架,并分别从权重优化、超参数优化和结构优化方面分析了当前演化循环神经网络的研究进展;然后,对演化循环神经网络的一些其他工作进行了分析;最后,指出了演化循环神经网络面临的挑战以及发展趋势。
Evolutionary computation utilizes natural selection mechanisms and genetic laws in the process of biological evolution to solve optimization problems.The accuracy and efficiency of the evolutionary recurrent neural network model depends on the optimization effect of parameters and the structures.The utilization of evolutionary computation to solve the problem of adaptive optimization of parameters and structures in recurrent neural networks is a hot spot of automated deep learning.This paper summarizes the algorithms that combine evolutionary algorithms and recurrent neural networks.Firstly,it briefly reviews the traditional categories,common algorithms,and advantages of evolutionary computation.Next,it briefly introduces the structures and characteristics of the recurrent neural network models and analyzes the influencing factors of recurrent neural network perfor-mance.Then,it analyzes the algorithmic framework of evolutionary recurrent neural networks,and the current research development of evolutionary recurrent neural networks from weight optimization,hyperparameter optimization and structure optimization.Besides,other work on evolutionary recurrent neural networks is analyzed.Finally,it points out the challenges and the deve-lopment trend of evolutionary recurrent neural networks.
作者
胡中源
薛羽
查加杰
HU Zhongyuan;XUE Yu;ZHA Jiajie(School of Software,Nanjing University of Information Science and Technology,Nanjing 210044,China)
出处
《计算机科学》
CSCD
北大核心
2023年第3期254-265,共12页
Computer Science
基金
国家自然科学基金(61876089)
数据科学与智慧软件江苏省重点实验室开放课题基金(2019DS302)
江苏省自然科学基金(BK20141005)
江苏省高校自然科学研究项目(14KJB520025)
江苏省研究生科研与实践创新计划(KYCX22_1206)。
关键词
循环神经网络
演化计算
权重优化
超参数优化
结构优化
集成学习
迁移学习
Recurrent neural network
Evolutionary computation
Weight optimization
Hyperparameter optimization
Optimization of structure
Ensemble learning
Transfer learning
