摘要
深度卷积神经网络在多个领域展现了不凡的性能,并被广泛应用.随着网络深度的增加和网络结构不断复杂化,计算资源和存储资源的需求也在不断攀升.专用硬件可以很好地解决对计算和存储的双重需求,在低功耗同时满足较高的计算性能,从而应用在一些无法使用通用CPU和GPU的场景中.在专用硬件设计过程中仍存在着很多亟待解决的问题,例如选择何种数据表示方法、如何平衡数据表示精度与硬件实现代价等.为解决上述问题,针对定点数和浮点数建立误差分析模型,从理论角度分析如何选择表示精度及选择结果对网络准确率的影响,并通过实验探究不同数据表示方法对硬件实现代价的影响.通过理论分析和实验验证可知,在一般情况下,满足同等精度要求时浮点表示方法在硬件实现开销上占有一定优势.除此之外,还根据浮点表示特征对神经网络中卷积操作进行了硬件实现,与定点数相比在功耗和面积上分别降低92.9%和77.2%.
Deep convolution neural networks have been widely used in industries as well as academicarea because of their triumphant performance.The reare tendencies toward deeper and more complexnet work structures,which leads to demand of substantial computation and memory resources.Customized hard ware is an appropriate and feasible option,which is beneficial to maintain highp erformance in lower e nergy consumption.Further more,customized h ard ware can also be adopted insome special situations where CPU and GPU cannot be placed.During the hardware-designingprocesses,we need to address some problems like how to choose different types of numericrepresentation as well as precision.In this article,we focus on two typical num e r i c representations,fixed-point a n d floating-point,a n d propose corresponding error models.Using these models,wetheoretically analyze the influence of different types of data representation o n the hard ware overhead ofneural networks.It is remarkable that floating-point has clear advantages over fixed-point underordinary circumstances.In general,we verify thr o u g h experiments that floating-point numbers,which are limited to certain precision,preponderate in both hardware area and power consumption.What’s more,according to the features of floating-point representation,our customized h a r d w a r eimplementation of convolution computation declines the power and area with14.1X a n d4.38Xrespectively.
作者
王佩琪
高原
刘振宇
王海霞
汪东升
Wang Peiqi;Gao Yuan;Liu Zhenyu;Wang Haixia;Wang Dongsheng(Department of Computer Science and Technology,Tsinghua University,Beijing 100084;Tsinghua National Laboratory for Information Science and Technology,Beijing 100084)
出处
《计算机研究与发展》
EI
CSCD
北大核心
2017年第6期1348-1356,共9页
Journal of Computer Research and Development
基金
国家自然科学基金项目(61373025)
国家重点研发计划项目(2016YFB1000303)~~
关键词
深度卷积神经网络
数据表示方式
浮点数据表示
定点数据表示
卷积操作优化
deep convolution neural network
numeric representation
floating-point computation
fixed-point computation
convolution optimization
