摘要
深度前馈神经网络在分类和回归问题上得到了很好的应用,但网络性能极大程度上受到其结构和超参数影响。为了获得高性能的神经网络,首先对遗传算法的选择策略进行改进,之后利用该改进遗传算法,采用二进制编码与实数编码的混合编码策略对深度前馈神经网络层数、每层节点量以及学习率和权重进行优化。改进的选择策略,在最优保存策略的基础上从父代和子代合并的2n个个体中,以一定的概率选择部分适应值较差个体作为新父代,以增加种群多样性,避免陷入局部最优。同时引入dropout方法减少网络过拟合训练数据。使用Ring、Breast cancer、Twonorm、Heart、Blood、Ionosphere、Monk共7个数据集进行数值实验,并与其他相关文献中的算法比较,仿真结果表明,改进的遗传算法能搜索到较高性能的神经网络。
Deep feed-forward neural networks are well applied in classification and regression problems,but network performance is greatly affected by their structure and hyper-parameters.To achieve high performance neural networks,a modified genetic algorithm is designed firstly,which modifies the selection strategy.Then,the modified genetic algorithm is employed to optimize the number of network layers,the number of nodes in each layer,and the learning rate and weights,which are coded by binary coding and real number coding strategy respectively.For the modified selection strategy,in 2n indivi-duals from the combination of parent population with offspring population,some top fitness individuals are selected and some worse fitness individuals with a high probability are also selected to achieve better diversity and avoid falling into local optimum.dropout method is introduced to avoid the overfitting training data of network.Seven datasets(Ring,Breast cancer,Twonorm,Heart,Blood,Ionosphere,Monk)are used in the experiments.The results show that,compared with the algorithms in related literatures,the modified genetic algorithm has higher performance neural networks.
作者
李静
莫思敏
LI Jing;MO Si-min(School of Economics and Management,Taiyuan University of Science and Technology,Taiyuan 030024,China)
出处
《计算机工程与科学》
CSCD
北大核心
2021年第8期1503-1511,共9页
Computer Engineering & Science
基金
国家青年科学基金(61703297)
山西省高等学校人文社会科学重点研究基地项目(20200129)。
关键词
深度前馈神经网络
改进遗传算法
网络结构优化
超参数优化
deep feed-forward neural network
modified genetic algorithm
network structure optimization
hyper-parameter optimization
