摘要
深度学习中,多模态模型的训练通常需要大量高质量不同类型的标注数据,如图像、文本、音频等.然而,获取大规模的多模态标注数据是一项具有挑战性和昂贵的任务.为了解决这一问题,主动学习作为一种有效的学习范式被广泛应用,能够通过有针对性地选择最有信息价值的样本进行标注,从而降低标注成本并提高模型性能.现有的主动学习方法往往面临着低效的数据扫描和数据位置调整问题,当索引需要进行大范围的更新时,会带来巨大的维护代价.为解决这些问题,提出了一种面向多模态模型训练的高效样本检索技术So-CBI.该方法通过感知模型训练类间边界点,精确评估样本对模型的价值;设计了半有序的高效样本索引,通过结合数据排序信息和部分有序性,降低了索引维护代价和时间开销.在多组多模态数据集上通过与传统主动学习训练方法实验对比,验证了So-CBI方法在主动学习下的训练样本检索问题上的有效性.
Training multimodal models in deep learning often requires a large amount of high-quality annotated data from diverse modalities such as images,text,and audio.However,acquiring such data in large quantities can be challenging and costly.Active learning has emerged as a powerful paradigm to address this issue by selectively annotating the most informative samples,thereby reducing annotation costs and improving model performance.However,existing active learning methods encounter limitations in terms of inefficient data scanning and costly maintenance when dealing with large-scale updates.To overcome these challenges,this study proposes a novel approach called So-CBI(semi-ordered class boundary index)that efficiently retrieves samples for multimodal model training.So-CBI incorporates inter-class boundary perception and a semi-ordered indexing structure to minimize maintenance costs and enhance retrieval efficiency.Experimental evaluations on various datasets demonstrate the effectiveness of So-CBI in the context of active learning.
作者
唐秀
伍赛
侯捷
陈刚
TANG Xiu;WU Sai;HOU Jie;CHEN Gang(College of Software,Zhejiang University,Ningbo 315103,China;College of Computer Science and Technology and College of Software,Zhejiang University,Hangzhou 310027,China)
出处
《软件学报》
EI
CSCD
北大核心
2024年第3期1125-1139,共15页
Journal of Software
基金
国家重点研发计划(2022YFB3304100)。
关键词
多模态模型训练
主动学习
样本检索
multimodal model training
active learning
sample retrieval
