摘要
针对在无约束环境下静态手势在识别过程中准确率不高的问题,本文提出了一种融合手部骨架灰度图(Grayscale Image of Hand Skeleton,GHS)的深度神经网络,使用手部关键点及其相互关联性构建手部骨架灰度图。网络的输入为GHS图像和RGB图像,主干网络为yolov3,添加了扩展卷积残差模块,在GHS图像和RGB图像进行特征融合后,通过SE模块对每个通道上的特征进行缩放,采用RReLU激活函数来代替Leaky ReLU激活函数。通过手部关键点及其相互间的连接信息增强手部图像特征,增大手势的类间差异,同时降低无约束环境对手势识别的影响,以提高手势识别的准确率。实验结果表明,在Microsoft Kinect&Leap Motion数据集上相比其他方法,本文方法的平均准确率达到最高,为99.68%;在Creative Senz3D数据集上相比其他方法,本文方法平均准确率达到最高,为99.8%。
In order to solve the problem of low accuracy of static gestures in the recognition process under unconstrained environment,this paper proposes a deep neural network combined with Grayscale Image of Hand Skeleton(GHS),using the key points of the hand and their correlations to build a grayscale image of the hand skeleton.The input of the network is GHS image and RGB image,the backbone network is yolov3,and the extended convolution residual module is added.After the feature fusion of the GHS image and the RGB image,the feature on each channel is scaled by the SE module,using RReLU Activation function to replace Leaky ReLU activation function.Through the key points of the hand and the connection information between them,the image features of the hand are enhanced,the difference between gestures is increased,and the influence of the unconstrained environment on gesture recognition is reduced to improve the accuracy of gesture recognition.The experimental results show that compared with other methods on the Microsoft Kinect&Leap Motion data set,the average accuracy rate of the method in this paper reaches the highest,99.68%;on the Creative Senz3 D data set,the average accuracy rate of the method in this paper reaches the highest,99.8%.
作者
章东平
束元
周志洪
ZHANG Dongping;SHU Yuan;ZHU Zhihong(Key Laboratory of Electromagnetic Wave Information Technology and Metrology of Zhejiang Province,College of Information,China Jiliang University,Hangzhou Zhejiang 310018,China;Shanghai Key Laboratory of Integrated Administration Technologies for Information Security,Shanghai 200240,China)
出处
《传感技术学报》
CAS
CSCD
北大核心
2021年第2期203-210,共8页
Chinese Journal of Sensors and Actuators
基金
浙江省重点研发计划项目(2020C03104)
上海市信息安全综合管理技术研究重点实验室开放课题(AGK201900X)。
