摘要
目的针对目前足迹检索中存在的采集设备种类多样化、有效的足迹特征难以提取等问题,本文以赤足足迹图像为研究对象,提出一种基于非局部(non-local)注意力双分支网络的跨模态赤足足迹检索算法。方法该网络由特征提取、特征嵌入以及双约束损失模块构成,其中特征提取模块采用双分支结构,各分支均以Res Net50作为基础网络分别提取光学和压力赤足图像的有效特征;同时在特征嵌入模块中通过参数共享学习一个多模态的共享空间,并引入非局部注意力机制快速捕获长范围依赖,获得更大感受野,专注足迹图像整体压力分布,在增强每个模态有用特征的同时突出了跨模态之间的共性特征;为了增大赤足足迹图像类间特征差异和减小类内特征差异,利用交叉熵损失LCE(cross-entropy loss)和三元组损失LTRI(triplet loss)对整个网络进行约束,以更好地学习跨模态共享特征,减小模态间的差异。结果本文将采集的138人的光学赤足图像和压力赤足图像作为实验数据集,并将本文算法与细粒度跨模态检索方法FGC(fine-grained cross-model)和跨模态行人重识别方法HC(hetero-center)进行了对比实验,本文算法在光学到压力检索模式下的m AP(mean average precision)值和rank1值分别为83.63%和98.29%,在压力到光学检索模式下的m AP值和rank1值分别为84.27%和94.71%,两种检索模式下的m AP均值和rank1均值分别为83.95%和96.5%,相较于FGC分别提高了40.01%和36.50%,相较于HC分别提高了26.07%和19.32%。同时本文算法在non-local注意力机制、损失函数、特征嵌入模块后采用的池化方式等方面进行了对比分析,其结果证实了本文算法的有效性。结论本文提出的跨模态赤足足迹检索算法取得了较高的精度,为现场足迹比对、鉴定等应用提供了研究基础。
ObjectiveFootprints are the highest rate of material evidence left and extracted from crime scene in general.Footprint retrieval and comparison plays an important role in criminal investigation.Footprint features are identified via the foot shape and bone structure of the person involved and have its features of specificity and stability.Meanwhile,footprints can reveal their essential behavior in the context of the physiological and behavioral characteristics.It is related to the biological features like height,body shape,gender,age and walking habits.Medical research results illustrates that footprint pressure information of each person is unique.It is challenged to improve the rate of discovery,extraction and utilization of footprints in criminal investigation.The retrieval of footprint image is of great significance,which will provide theoretical basis and technical support for footprint comparison and identification.Footprint images have different modes due to the diverse scenarios and tools of extraction.The global information of cross-modal barefoot images is unique,which can realize retrieval-oriented.The retrieval orientation retrieves the corresponding image of cross-modes.The traditional cross-modal retrieval methods are mainly in the context of subspace method and objective model method.These retrieval methods are difficult to obtain distinguishable features.The deep learning based retrieval methods construct multi-modal public space via convolutional neural network(CNN).The high-level semantic features of image can be captured in terms of iterative optimization of network parameters,to lower the multi-modal heterogeneity.MethodA cross-modal barefoot footprint retrieval algorithm based on non-local attention two-branch network is demonstrated to resolve the issue of intra-class wide distance and inter-class narrow distance in fine-grained images.The collected barefoot footprint images involve optical mode and pressure mode.The median filter is applied to remove noises for all images,and the data augmen
作者
鲍文霞
茅丽丽
王年
唐俊
杨先军
张艳
Bao Wenxia;Mao Lili;Wang Nian;Tang Jun;Yang Xianjun;Zhang Yan(College of Electronic Information Engineering,Anhui University,Hefei 230601,China;Hefei Institutes of Physical Science,Chinese Academy of Sciences,Hefei 230031,China)
出处
《中国图象图形学报》
CSCD
北大核心
2022年第7期2199-2213,共15页
Journal of Image and Graphics
基金
国家重点研发计划资助(2020YFF0303803)
国家自然科学基金项目(61772032)
安徽高校自然科学研究重点项目(KJ2021ZD0004,KJ2019A0027)。
关键词
图像检索
跨模态足迹检索
非局部注意力机制
双分支网络
赤足足迹图像
image retrieval
cross-modal footprint retrieval
non-local attention mechanism
two-branch network
barefoot footprint image
