为解决目前深度仿造检测方法对于跨数据集的检测性能难以提高的问题,提出基于注意力机制和一致性损失相结合的深度伪造人脸检测方法(method based on attention mechanism and consistency loss,MAMCL)。采用多注意力机制,迫使网络捕捉...为解决目前深度仿造检测方法对于跨数据集的检测性能难以提高的问题,提出基于注意力机制和一致性损失相结合的深度伪造人脸检测方法(method based on attention mechanism and consistency loss,MAMCL)。采用多注意力机制,迫使网络捕捉到更细微的局部异常。采用基于注意力机制的擦除方式,鼓励模型深入挖掘之前忽略的区域。设计一致性模块获取伪造图像中普遍存在的不一致细节特征,并应用一致性损失引导模型更加关注伪造细节。在面部取证++(FaceForensics++,FF++)数据集上进行实验,准确率达到96.38%,受试者工作特征曲线(receiver operating characteristic curve,ROC)的曲线下面积达到99.34%,在泛化性能测试中也取得了良好的效果。通过消融实验,证明了每个模块的有效性。结果表明,提出的检测方法能够较为准确地检测深度伪造人脸,且具有良好的泛化性能,可以作为应对当前人脸伪造威胁的有效检测手段。展开更多
面部表情是传递情感的重要信息,是家畜生理、心理和行为的综合反映,可以用于评估家畜福利。由于家畜面部肌群结构简单,因此家畜面部不同区域的细微变化对于表情的反映较难识别。该研究提出一种基于多注意力机制级联LSTM框架模型(Multi-a...面部表情是传递情感的重要信息,是家畜生理、心理和行为的综合反映,可以用于评估家畜福利。由于家畜面部肌群结构简单,因此家畜面部不同区域的细微变化对于表情的反映较难识别。该研究提出一种基于多注意力机制级联LSTM框架模型(Multi-attention Cascaded Long Short Term Memory,MA-LSTM)对家猪时序面部表情进行分类识别。首先通过简化的多任务级联卷积结构实现帧图像中猪脸的快速检测与定位,去除非猪脸区域对于识别性能的影响。其次提出一种多注意力机制模块,利用不同特征通道视觉信息不同相应峰值响应区域也不同这一特性,通过对峰值响应相近区域进行聚类捕获表情变化引起的面部显著性区域,实现对面部细微变化的关注。在自标注构建的家猪表情数据集上的试验结果表明,该研究提出的多注意力机制级联LSTM模型在4类表情的平均识别准确率为91.826%,对比关闭多注意力机制模块平均识别准确率平均提升6.3个百分点,同时误分率也有较为明显的降低。对比其他常用面部表情识别算法LBP-TOP、HOG-TOP、ELRCN、STC-NLSTM,MA-LSTM模型平均识别精度分别提升约32.6、18.0、5.9和4.4个百分点。试验结果验证了该研究提出的多注意力机制级联LSTM模型在猪脸表情识别的有效性。展开更多
Facial attribute editing has mainly two objectives:1)translating image from a source domain to a target one,and 2)only changing the facial regions related to a target attribute and preserving the attribute-excluding d...Facial attribute editing has mainly two objectives:1)translating image from a source domain to a target one,and 2)only changing the facial regions related to a target attribute and preserving the attribute-excluding details.In this work,we propose a multi-attention U-Net-based generative adversarial network(MU-GAN).First,we replace a classic convolutional encoder-decoder with a symmetric U-Net-like structure in a generator,and then apply an additive attention mechanism to build attention-based U-Net connections for adaptively transferring encoder representations to complement a decoder with attribute-excluding detail and enhance attribute editing ability.Second,a self-attention(SA)mechanism is incorporated into convolutional layers for modeling long-range and multi-level dependencies across image regions.Experimental results indicate that our method is capable of balancing attribute editing ability and details preservation ability,and can decouple the correlation among attributes.It outperforms the state-of-the-art methods in terms of attribute manipulation accuracy and image quality.Our code is available at https://github.com/SuSir1996/MU-GAN.展开更多
文摘为解决目前深度仿造检测方法对于跨数据集的检测性能难以提高的问题,提出基于注意力机制和一致性损失相结合的深度伪造人脸检测方法(method based on attention mechanism and consistency loss,MAMCL)。采用多注意力机制,迫使网络捕捉到更细微的局部异常。采用基于注意力机制的擦除方式,鼓励模型深入挖掘之前忽略的区域。设计一致性模块获取伪造图像中普遍存在的不一致细节特征,并应用一致性损失引导模型更加关注伪造细节。在面部取证++(FaceForensics++,FF++)数据集上进行实验,准确率达到96.38%,受试者工作特征曲线(receiver operating characteristic curve,ROC)的曲线下面积达到99.34%,在泛化性能测试中也取得了良好的效果。通过消融实验,证明了每个模块的有效性。结果表明,提出的检测方法能够较为准确地检测深度伪造人脸,且具有良好的泛化性能,可以作为应对当前人脸伪造威胁的有效检测手段。
文摘命名实体识别(named entity recognition,NER)是自然语言处理中重要的基础任务,而中文命名实体识别(Chinese named entity recognition,CNER)因分词歧义和一词多义等问题使其尤显困难。针对这些问题,提出多头注意力机制(multi-heads attention mechanism,Multi-Attention)与字词融合的中文命名实体识别模型(CWA-CNER)。将汉语文本字向量与其在句中可能成词的词向量进行拼接,并将其送入长短时记忆网络(bidirectional long short-term memory neural network,BiLSTM)提取上下文语义信息,进而利用多头注意力机制捕获句中元素间联系的紧密程度,最后通过条件随机场(conditional random field,CRF)进行实体标注。该模型在Boson数据集,1998和2014年《人民日报》三种语料上进行实验,其F1值均达到90%以上,结果表明了模型的有效性。
文摘面部表情是传递情感的重要信息,是家畜生理、心理和行为的综合反映,可以用于评估家畜福利。由于家畜面部肌群结构简单,因此家畜面部不同区域的细微变化对于表情的反映较难识别。该研究提出一种基于多注意力机制级联LSTM框架模型(Multi-attention Cascaded Long Short Term Memory,MA-LSTM)对家猪时序面部表情进行分类识别。首先通过简化的多任务级联卷积结构实现帧图像中猪脸的快速检测与定位,去除非猪脸区域对于识别性能的影响。其次提出一种多注意力机制模块,利用不同特征通道视觉信息不同相应峰值响应区域也不同这一特性,通过对峰值响应相近区域进行聚类捕获表情变化引起的面部显著性区域,实现对面部细微变化的关注。在自标注构建的家猪表情数据集上的试验结果表明,该研究提出的多注意力机制级联LSTM模型在4类表情的平均识别准确率为91.826%,对比关闭多注意力机制模块平均识别准确率平均提升6.3个百分点,同时误分率也有较为明显的降低。对比其他常用面部表情识别算法LBP-TOP、HOG-TOP、ELRCN、STC-NLSTM,MA-LSTM模型平均识别精度分别提升约32.6、18.0、5.9和4.4个百分点。试验结果验证了该研究提出的多注意力机制级联LSTM模型在猪脸表情识别的有效性。
基金supported in part by the National Natural Science Foundation of China(NSFC)(62076093,61871182,61302163,61401154)the Beijing Natural Science Foundation(4192055)+3 种基金the Natural Science Foundation of Hebei Province of China(F2015502062,F2016502101,F2017502016)the Fundamental Research Funds for the Central Universities(2020YJ006,2020MS099)the Open Project Program of the National Laboratory of Pattern Recognition(NLPR)(201900051)The authors gratefully acknowledge the support of NVIDIA Corporation with the donation of the GPU used for this research.
文摘Facial attribute editing has mainly two objectives:1)translating image from a source domain to a target one,and 2)only changing the facial regions related to a target attribute and preserving the attribute-excluding details.In this work,we propose a multi-attention U-Net-based generative adversarial network(MU-GAN).First,we replace a classic convolutional encoder-decoder with a symmetric U-Net-like structure in a generator,and then apply an additive attention mechanism to build attention-based U-Net connections for adaptively transferring encoder representations to complement a decoder with attribute-excluding detail and enhance attribute editing ability.Second,a self-attention(SA)mechanism is incorporated into convolutional layers for modeling long-range and multi-level dependencies across image regions.Experimental results indicate that our method is capable of balancing attribute editing ability and details preservation ability,and can decouple the correlation among attributes.It outperforms the state-of-the-art methods in terms of attribute manipulation accuracy and image quality.Our code is available at https://github.com/SuSir1996/MU-GAN.
