摘要
针对行为识别中时空信息分布不均衡以及对长时间跨度信息表征获取难的问题,提出了一种时间-动作感知的多尺度时间序列实时行为识别方法MRTP。以RGB视频为输入,使用两个并行的感知路径在不同的时间分辨率上对视频进行空间特征与动作特征提取。在空间路径中,使用基于特征差分的动作感知寻找并加强通道动作特征表征;在动作路径中,基于动作感知的权重对通道进行筛选,并加入通道注意力和时间注意力加强关键特征;在两个路径提取出特征后,对特征进行融合,融合后的特征通过激活函数映射出样本在各个类别的得分,取得分最高的类别为最终识别结果。实验结果表明:所提方法在UCF101数据集上达到了95.6%的准确率,优于未使用时间注意力的方法;在AVA2.2数据集上的平均精度达到了28%,优于未使用动作感知和时间注意力的方法。与目前主流的基于光流法的双流网络、以Slowfast为代表的3D卷积网络、Transformer等方法进行了准确率、参数量、处理速度对比,结果表明所提方法具有更良好的识别效果和鲁棒性。
In view of the uneven distribution of temporal and spatial information and the difficulty in obtaining long-term information representation,we propose a dual-path action recognition method MRTP based on time and motion perception,which takes RGB video as input and applies two parallel perception paths to extract spatial and motion features from the video at different time resolutions.In the spatial path,the action-perception based on feature difference is applied to find and strengthen the action feature representation;in the action path,the channel is filtered based on the weight of action perception,and channel attention and time attention are added to enhance the key features;the characteristics of the two paths are merged to calculate the action category score of the video.The experimental results show that MRTP achieves accuracy of 95.6%on the UCF101 data set to outperform the model without time attention;on the AVA2.2 data set,the accuracy of mAP reaches 28%to outperform the model without action perception and time attention.Compared with the current mainstream two-stream network,3D convolution,Transformer,and other methods on a number of accuracy indicators,this method is endowed with better recognition effect and robustness.
作者
张坤
杨静
张栋
陈跃海
李杰
杜少毅
ZHANG Kun;YANG Jing;ZHANG Dong;CHEN Yuehai;LI Jie;DU Shaoyi(School of Automation Science and Engineering, Xi’an Jiaotong University, Xi’an 710049, China;College of Artificial Intelligence, Xi’an Jiaotong University, Xi’an 710049, China)
出处
《西安交通大学学报》
EI
CAS
CSCD
北大核心
2022年第3期22-32,共11页
Journal of Xi'an Jiaotong University
基金
国家自然科学基金资助项目(62073257)。
关键词
行为识别
双路径网络
特征差分
动作感知
时间注意力
action recognition
dual-path network
feature difference
action perception
temporal attention
