摘要
因果关系普遍存在于群体交互行为中,体现出主动体行为对被动体行为的有向影响.因果关系检测的难点在于交互双方的行为具有复杂的时间动态性.现有方法使用循环神经网络,来描述交互关系的时间变化特性,并使用时间注意力机制,来描述时间依赖关系.上述方法忽视了对多人依赖关系的分析,难以区分交互双方中的主动行为者和被动行为者.本文设计了一种基于Granger因果关系的时空图推理模型,来学习交互双方的主动和被动关系.为了实现Granger因果关系检测,该模型对单个个体时序特征进行自回归建模,来描述行为对个体自己的依赖.该模型对两个个体时序特征进行相关回归建模,来描述行为对两个个体的依赖.该模型通过比较自回归误差和相关回归误差,当自回归误差明显大于相关回归误差,则说明相关个体改变了另一方个体的行为特征,从而检测出相关个体为主动个体,另一方为被动个体.相关回归模型考虑了多种时间延迟量的两个个体的时序特征序列,用于学习两个个体之间行为的时间延迟量.该时间延迟量用于将主动个体时间特征与被动个体时间特征进行对齐.时间对齐后的主动个体特征提供了被动个体的时间和空间上下文特征,并与被动个体特征进行通道级的融合.为了充分描述个体之间的外观模式,位置约束,因果关系的交互关系,该模型构建多尺度外观的因果图,并使用图推理学习融合上下文的个体特征和群体特征.本文对Granger因果关系检测进行消融分析,并说明时间延迟量,交互融合通道比例,多尺度图推理,能够有效改善个体特征、群体特征的描述能力.本文方法在Volleyball和Collective Activity数据集上优于现有群体行为识别方法.本文的可视化结果说明Granger因果关系可以捕获群体中关键的交互关系.
Causality reflects the directional effect from the active actor to the passive actor and commonly exists in group interactions.The difficulty in causality detection lies in the complex temporal dynamics of sequential features of the interacting actors.Existing methods use recurrent neural networks to describe the temporal dynamics of the interaction relations.Some methods use temporal attention mechanisms to describe temporal dependencies.They neglect to analyze the dependency between two actors,and are hard to distinguish the active actor and passive actor in the interaction.In this work,we design a Granger causality-based spatiotemporal graph model to learn the active-passive relations between interacting actors.To detect the Granger causality,the model designs an autoregression function for single individual temporal sequential features to describe the dependence of action on the individual itself.The model designs a correlative regression function for two individual temporal sequential features to describe the dependence of action on two individuals.The model detects the correlative individual as an active individual and the other as a passive individual by comparing the autoregressive error with the correlative regression error,when the autoregressive error is significantly larger than the correlative regression error,which indicates that the correlative individuals change the action of the other individual.The correlative regression function considers two individual temporal sequential features with multiple time delays,which can be used to learn the amount of time delay for actions between two individuals.This time delay amount is used to align the active individual time features with the passive individual time features.The temporally aligned active individual features provide the temporal and spatial contextual features of the passive individual and are fused with the passive individual features at the channel-wise level.The model constructs causal graphs of multi-scale spatiotemporal features to fully d
作者
谢昭
李骏
吴克伟
焦畅
XIE Zhao;LI Jun;WU Ke-Wei;JIAO Chang(Key Laboratory of Knowledge Engineering with Big Data,Hefei University of Technology,Ministry of Education,Hefei 230601;Anhui Province Key Laboratory of Industry Safety and Emergency Technology(Hefei University of Technology),Hefei 230601;School of Computer Science and Information Engineering,Hefei University of Technology,Hefei 230601)
出处
《计算机学报》
EI
CAS
CSCD
北大核心
2023年第4期856-876,共21页
Chinese Journal of Computers
基金
安徽省重点研究与开发计划(202004d07020004)
安徽省自然科学基金项目(2108085MF203)
中央高校基本科研业务费专项资金资助(PA2021GDSK0072,JZ2021HGQA0219)资助.
关键词
群体行为识别
GRANGER因果关系
时间延迟依赖
时空上下文
图卷积推理
group activity recognition
Granger causality relation
temporal delay dependency
spatial-temporal context
graph convolutional inference
