摘要
现有跨模态检索方法主要使用某一指标约束得到一个子空间,检索结果往往有差异.为了提高公共子空间的鲁棒性,文中提出基于关联约束的对抗跨模态检索方法.对抗约束通过混淆判别器使其无法分辨子空间特征来自哪个模态,从而提升不同模态特征的一致性.关联约束用于增强投影子空间关联程度.三元组约束同时考虑不同模态同一语义、相同模态不同语义样本之间的结构信息.在数据集上的实验表明文中方法的检索性能得到有效提升.
In the existing cross-modal retrieval methods,retrieval results are obtained via the subspace acquired by a certain index constraint such as distance or similarity.Since the subspaces are learned with different index constraints,retrieval results are different.To improve the robustness of common subspace,a method for adversarial cross-modal retrieval based on association constraint is proposed.The consistency of different modality features is improved by the adversarial constraint to make the discriminator in the constraint unable to distinguish which modality the subspace features come from.The association of different modality features is enhanced by the association constraint.The structural information between example pairs with the same semantics of different modalities and different semantics of the same modality is taken into account by the triple loss constraint.Experimental results on datasets show that the proposed method is more effective than other cross-modal retrieval methods.
作者
郭倩
钱宇华
梁新彦
GUO Qian;QIAN Yuhua;LIANG Xinyan(Institute of Big Data Science and Industry,Shanxi University,Taiyuan 030006;Key Laboratory Computational Intelligence and Chinese Information Processing of Ministry of Education,Shanxi University,Taiyuan 030006;School of Computer and Information Technology,Shanxi University,Taiyuan 030006)
出处
《模式识别与人工智能》
EI
CSCD
北大核心
2021年第1期68-76,共9页
Pattern Recognition and Artificial Intelligence
基金
国家自然科学基金项目(No.61672332,61802238,61603228,62006146,61906115,F060308)
山西省重点研发计划(国际科技合作)项目(No.201903D421003)
山西省拔尖创新人才支持计划、山西省三晋学者、山西省回国留学人员科研项目(No.2017023,2018172,HGKY2019001)
山西省青年基金项目(No.201901D211171,201901D211169)
山西省高等学校科技创新项目(No.2020L0036)资助。
关键词
跨模态检索
多指标约束
对抗约束
关联约束
Cross-Modal Retrieval
Multi-index Constraint
Adversarial Constraint
Association Constraint
