摘要
针对神经网络结构设计问题,提出一种基于神经网络复杂度的修剪算法.其实质是在训练过程中,利用网络连接权矩阵的协方差矩阵计算网络的信息熵,获得网络的复杂度;在保证网络信息处理能力的前提下,删除对网络复杂度影响最小的隐节点.该算法不要求训练网络到代价函数的极小点,适合在线修剪网络结构,并且避免了结构调整前的网络权值预处理.通过对典型函数逼近的实验结果表明,该算法在保证网络逼近精度的同时,可有效地简化网络结构.
For the design of the neural network architecture,a pruning algorithm based on the neural complexity is proposed.The essence is to calculate the entropy of neural network by the standard covariance matrix of the neural network＇s connection matrix in the process of training,and the network＇s complexity can be acquired.In the premise of ensuring the information processing capacity of neural network,the least important hidden node is deleted.It is not necessary to train the cost function of the neural network to a local minimal,suitable for pruning neural network architecture on-line,and the pre-processing neural network weights are avoided before architecture adjustment of the neural network.The simulation results of the typical function approximation show that the precision of the approximation is ensured and at the same time a simple architecture of neural networks can be achieved.
出处
《控制与决策》
EI
CSCD
北大核心
2010年第6期821-824,830,共5页
Control and Decision
基金
国家自然科学基金项目(60873043)
国家863计划项目(2007AA04Z160
2009AA04Z155)
教育部博士点基金项目(200800050004)
北京市自然科学基金项目(4092010)
关键词
修剪算法
神经复杂度
互信息熵
Pruning algorithm
Neural complexity
Mutual information entropy
