摘要
随着分段线性函数的广泛应用,本文尝试研究浅层和深层的分段线性神经网络的逼近理论.作者将应用于三层感知机模型的万能逼近定理拓展到分段线性神经网络中,并给出与隐藏神经元个数相关的逼近误差估计.利用分段线性函数构造锯齿函数的显式方法,证明解析函数可以通过分段线性神经网络的深度堆叠以指数速率逼近,并辅以相应的数值实验.
With the wide application of Piece Wise Linear(PWL for short)functions,this paper attempts to address the approximation theory on Piece Wise Linear Neural Networks(PWLNNs for short)for both shallow networks and deep neural networks(DNNs for short).The authors extend the universal approximation theorem of three-layer MultiLayer Perceptrons(MLPs for short)with PWL functions and bound the error by the number of hidden neurons.The authors give an explicit way of constructing sawtooth functions from PWL functions,and thus prove analytic functions can be approximated at an exponentially convergent rate by stacking depth rather than increasing width.Numerical experiments are also provided to verify the conclusions.
作者
吴心宇
陈天平
卢文联
WU Xinyu;CHEN Tianping;LU Wenlian(School of Mathematical Sciences,Fudan University,Shanghai 200433,China;Shanghai Center for Mathematical Sciences,Fudan University,Shanghai 200438,China;Shanghai Key Laboratory for Contemporary Applied Mathematics,Shanghai 200433,China)
出处
《数学年刊（A辑）》
CSCD
北大核心
2024年第1期53-70,共18页
Chinese Annals of Mathematics
基金
国家重点研发计划(No.2018AAA0100303)
国家自然科学基金(No.62072111)
上海市科技重大专项(No.2018SHZDZX01)
张江实验室项目的资助。
关键词
分段线性
神经网络
逼近理论
Piece wise linear
Neural network
Approximation theorem
