摘要
目的传统的手绘图像检索方法主要集中在检索相同类别的图像,忽略了手绘图像的细粒度特征。对此,提出了一种新的结合细粒度特征与深度卷积网络的手绘图像检索方法,既注重通过深度跨域实现整体匹配,也实现细粒度细节匹配。方法首先构建多通道混合卷积神经网络,对手绘图像和自然图像分别进行不同的处理;其次通过在网络中加入注意力模型来获取细粒度特征;最后将粗细特征融合,进行相似性度量,得到检索结果。结果在不同的数据库上进行实验,与传统的尺度不变特征(SIFT)、方向梯度直方图(HOG)和深度手绘模型Deep Sa N(sketch-a-net)、Deep 3DS(sketch)、Deep TSN(triplet sketch net)等5种基准方法进行比较,选取了Top-1和Top-10,在鞋子数据集上,本文方法 Top-1正确率提升了12%,在椅子数据集上,本文方法 Top-1正确率提升了11%,Top-10提升了3%,与传统的手绘检索方法相比,本文方法得到了更高的准确率。在实验中,本文方法通过手绘图像能在第1幅检索出绝大多数的目标图像,达到了实例级别手绘检索的目的。结论提出了一种新的手绘图像检索方法,为手绘图像和自然图像的跨域检索提供了一种新思路,进行实例级别的手绘检索,与原有的方法相比,检索精度得到明显提升,证明了本文方法的可行性。
Objective Content-based image retrieval or text-based retrieval has played a major role in practical computer vision applications. In several scenarios, however, retrieval becomes a problem when sample queries are unavailable or describing them with a keyword is difficult. However, compared with text, sketches can intrinsically capture object appearance and structure. Sketches are incredibly intuitive to humans and descriptive in nature. They provide a convenient and intuitive means to specify object appearance and structure. As a query modality, they offer a degree of precision and flexibility that is missing in traditional text-based image retrieval. Closely correlated with the proliferation of touch-screen devices, sketch-based image retrieval has become an increasingly prominent research topic in recent years. Conventional sketch-based image retrieval (SBIR) principally focuses on retrieving images of the same category and disregards the fine-grained feature of sketches. However, SBIR is challenging because humans draw free-hand sketches without any reference but only focus on the salient object structures. Hence, the shapes and scales in sketches are usually distorted compared with those in natural images. To deal with this problem, studies have developed methods to bridge the domain gap between sketches and natural images for SBIR. These approaches can be roughly divided into hand-crafted and cross-domain deep learning-based methods. SBIR generates approximate sketches by extracting edge or contour maps from natural images. Afterward, hand-crafted features are extracted for sketches and edge maps of natural images, which are then fed into "bag-of-words" methods to generate representations for SBIR. The major limitation of hand-crafted methods is that the domain gap between sketches and natural images cannot be well remedied because matching edge maps to non-aligned sketches with large variations and ambiguity is difficult. For this problem, we propose a novel sketch-based image retrieval method based on fin
作者
李宗民
刘秀秀
刘玉杰
李华
Li Zongmin;Liu Xiuxiu;Liu Yujie;Li Hua(College of Computer and Communication Engineering,China University of Petroleum,Qingdao 266580,China;Key Laboratory of Intelligent Information Processing,Institute of Computing Technology,Chinese Academy of Sciences,Beijing 100190,China)
出处
《中国图象图形学报》
CSCD
北大核心
2019年第6期946-955,共10页
Journal of Image and Graphics
基金
国家自然科学基金项目(61379106,61379082,61227802)
山东省自然科学基金项目(ZR2013FM036,ZR2015FM011)~~
关键词
手绘图像检索
卷积神经网络
注意力模型
细粒度特征
特征融合
sketch-based image retrieval (SBIR)
convolutional neural network
attention model
fine-grained feature
feature fusion
