摘要
注意力机制的出现和应用在一定程度上改善了神经网络对全局信息应用不足的缺陷,但常见的注意力机制模块也同样存在感受野小无法关注全局信息的问题,而某些全局注意力机制模块则计算成本过高。为此,提出一种基于卷积、池化、对比方法的轻量化注意力模块,即全局采样空间注意力模块。对于深度网络推理过程中部分模块输出的中间特征图,该注意力模块通过对比差值的形式获取所需要的空间注意力图。全局采样空间注意力模块是一种轻量化的通用模块,能够直接置入卷积神经网络中,增加的成本几乎可以忽略不计,并且其能够与网络一同进行端到端训练。主要在随机抽取的部分ImageNet-1K数据集和团队自制的“低慢小”无人机数据集中对模块进行了验证。实验结果显示,相比其他模块,所提模块在图像分类和小目标检测识别任务中具备1百分点~3百分点的性能提升效果,证明了所提模块的性能与其在小目标检测方面的适用性。
The emergence and application of attention mechanisms have addressed some limitations of neural networks concerning the utilization of global information.However,common attention modules face issues with the receptive field being too small to focus on overall information.Moreover,existing global attention modules tend to incur high computational costs.To address these challenges,a lightweight,universal attention module,termed"global-sampling spatial-attention module",is introduced herein based on convolution,pooling,and comparison methods.This module relies on the comparison methods to derive spatial-attention maps for intermediate feature maps generated during deep network inference.Moreover,this module can be directly integrated into convolutional neural networks with minimal costs and can be end-to-end trained with the networks.The introduced module was primarily validated using a randomly selected subset of the ImageNet-1K dataset and a proprietary low-slow-small drone dataset.Experimental results show that compared with other modules,this module exhibits an improvement of approximately 1-3 percentage points in tasks related to image classification and small object detection and recognition.These findings underscore the efficacy of the proposed module and its applicability in small object detection.
作者
卢镜宇
张海洋
王文鑫
赵长明
Lu Jjingyu;Zhang Haiyang;Wang Wenxin;Zhao Changming(School of Optics and Photonics,Beijing Institute of Technology,Beijing 100081;Key Laboratory of Optoelectronic Imaging Technology and Systems,Ministry of Education,Beijing 100081;Key Laboratory of Information Photonics Technology,Ministry of Industry and Information Technology,Beijing100081)
出处
《激光与光电子学进展》
CSCD
北大核心
2024年第10期396-407,共12页
Laser & Optoelectronics Progress
关键词
注意力机制
全局采样
轻量化
图像分类
小目标探测
attention mechanism
global sampling
lightweight
image classification
small target detection
