Handheld ultrasound devices are known for their portability and affordability,making them widely utilized in underdeveloped areas and community healthcare for rapid diagnosis and early screening.However,the image qual...Handheld ultrasound devices are known for their portability and affordability,making them widely utilized in underdeveloped areas and community healthcare for rapid diagnosis and early screening.However,the image quality of handheld ultrasound devices is not always satisfactory due to the limited equipment size,which hinders accurate diagnoses by doctors.At the same time,paired ultrasound images are difficult to obtain from the clinic because imaging process is complicated.Therefore,we propose a modified cycle generative adversarial network(cycleGAN) for ultrasound image enhancement from multiple organs via unpaired pre-training.We introduce an ultrasound image pre-training method that does not require paired images,alleviating the requirement for large-scale paired datasets.We also propose an enhanced block with different structures in the pre-training and fine-tuning phases,which can help achieve the goals of different training phases.To improve the robustness of the model,we add Gaussian noise to the training images as data augmentation.Our approach is effective in obtaining the best quantitative evaluation results using a small number of parameters and less training costs to improve the quality of handheld ultrasound devices.展开更多
基于预训练和注意机制的意图分类和语义槽填充,提出一种结合双向长短时记忆(bidirectional long short-term memory,BiLSTM)、条件随机场(conditional random fields,CRF)和注意机制的双向编码(bidirectional encoder representations f...基于预训练和注意机制的意图分类和语义槽填充,提出一种结合双向长短时记忆(bidirectional long short-term memory,BiLSTM)、条件随机场(conditional random fields,CRF)和注意机制的双向编码(bidirectional encoder representations from transformers,BERT)具有双向编码表示和注意机制的联合模型。该模型无需过多依赖手工标签数据和领域特定的知识或资源,避免了目前普遍存在的弱泛化能力。在自主公交信息查询系统语料库上进行的试验表明,该模型意图分类的准确性和语义槽填充F1值分别达到98%和96.3%,均产生有效改进。展开更多
人体目标检测对社会治理和城市安全具有很重要的现实意义,监控数据是数据安全的重要来源。小目标检测是目前受到广泛关注的安全检测问题中一项具有挑战性的任务,其检测对象为大型图像中少于20个像素的目标。小目标的特征难以表征,其中...人体目标检测对社会治理和城市安全具有很重要的现实意义,监控数据是数据安全的重要来源。小目标检测是目前受到广泛关注的安全检测问题中一项具有挑战性的任务,其检测对象为大型图像中少于20个像素的目标。小目标的特征难以表征,其中一个主要挑战是,用于预训练/共同训练检测器的数据集(如COCO)与用于微调检测器的数据集(如TinyPerson)之间存在尺度不匹配的情况,这给小目标检测器的性能带来了负面影响。为了解决这个问题,文中提出了一种优化策略,用于匹配不同数据集的尺度,称其为尺度分布搜索(Scale Distribution Search,SDS),同时平衡图片的信息收益(数据集之间的尺度相近)和信息损失(信噪比(SNR)的降低)。该策略使用高斯模型对数据集中目标的尺度分布进行建模,通过迭代的方式寻找最优分布参数;并对比数据集中目标的特征分布和检测器的性能,以找到最佳的尺度分布。通过SDS策略,主流目标检测方法在TinyPerson上实现了更好的性能,证明了SDS策略在提升预训练/共同训练效率上的有效性。展开更多
Graph neural networks(GNNs)have shown great power in learning on graphs.However,it is still a challenge for GNNs to model information faraway from the source node.The ability to preserve global information can enhance...Graph neural networks(GNNs)have shown great power in learning on graphs.However,it is still a challenge for GNNs to model information faraway from the source node.The ability to preserve global information can enhance graph representation and hence improve classification precision.In the paper,we propose a new learning framework named G-GNN(Global information for GNN)to address the challenge.First,the global structure and global attribute features of each node are obtained via unsupervised pre-training,and those global features preserve the global information associated with the node.Then,using the pre-trained global features and the raw attributes of the graph,a set of parallel kernel GNNs is used to learn different aspects from these heterogeneous features.Any general GNN can be used as a kernal and easily obtain the ability of preserving global information,without having to alter their own algorithms.Extensive experiments have shown that state-of-the-art models,e.g.,GCN,GAT,Graphsage and APPNP,can achieve improvement with G-GNN on three standard evaluation datasets.Specially,we establish new benchmark precision records on Cora(84.31%)and Pubmed(80.95%)when learning on attributed graphs.展开更多
近年来,网络安全威胁日益增多,数据驱动的安全智能分析成为网络安全领域研究的热点。特别是以知识图谱为代表的人工智能技术可为多源异构威胁情报数据中的复杂网络攻击检测和未知网络攻击检测提供支撑。网络安全实体识别是威胁情报知识...近年来,网络安全威胁日益增多,数据驱动的安全智能分析成为网络安全领域研究的热点。特别是以知识图谱为代表的人工智能技术可为多源异构威胁情报数据中的复杂网络攻击检测和未知网络攻击检测提供支撑。网络安全实体识别是威胁情报知识图谱构建的基础。开放网络文本数据中的安全实体构成非常复杂,导致传统的深度学习方法难以准确识别。在BERT(pre-training of deep bidirectional transformers)预训练语言模型的基础上,提出一种基于残差空洞卷积神经网络和条件随机场的网络安全实体识别模型BERT-RDCNN-CRF。通过BERT模型训练字符级特征向量表示,结合残差卷积与空洞神经网络模型有效提取安全实体的重要特征,最后通过CRF获得每一个字符的BIO标注。在所构建的大规模网络安全实体标注数据集上的实验表明,所提方法取得了比LSTM-CRF模型、BiLSTM-CRF模型和传统的实体识别模型更好的效果。展开更多
【目的】为了解决开源非结构化军事领域数据的命名实体识别问题。【方法】本文提出基于预训练模型(Bidirectional Encoder Representations from Transformers,BERT)的命名实体识别方法,首先基于预训练BERT模型生成自建开源军事语料的...【目的】为了解决开源非结构化军事领域数据的命名实体识别问题。【方法】本文提出基于预训练模型(Bidirectional Encoder Representations from Transformers,BERT)的命名实体识别方法,首先基于预训练BERT模型生成自建开源军事语料的动态特征词向量的字符表示,然后基于双向长短期记忆网络(Bi-directional Long Short-Term Memory,BiLSTM)完成语义特征提取,最后利用条件随机场模型(Conditional Random Fields,CRF)选出最优标签序列完成实体识别的任务。【结果】模型在自建的开源军事数据集上的实验结果表明,相较于基于统计模型和神经网络的方法,本文提出的方法可使准确率提升8%,F值提高11%,召回率提高10%。【局限】由于现阶段在开源军事领域中公开标注数据集较为缺乏,所以未能在开源军事语料上训练BERT模型。【结论】但本文提出的基于预训练模型的开源军事命名实体识别方法,在一定程度上解决了边界划分问题,同时解决了在数据集不足的情况下实体识别任务表现不佳的问题。展开更多
文摘Handheld ultrasound devices are known for their portability and affordability,making them widely utilized in underdeveloped areas and community healthcare for rapid diagnosis and early screening.However,the image quality of handheld ultrasound devices is not always satisfactory due to the limited equipment size,which hinders accurate diagnoses by doctors.At the same time,paired ultrasound images are difficult to obtain from the clinic because imaging process is complicated.Therefore,we propose a modified cycle generative adversarial network(cycleGAN) for ultrasound image enhancement from multiple organs via unpaired pre-training.We introduce an ultrasound image pre-training method that does not require paired images,alleviating the requirement for large-scale paired datasets.We also propose an enhanced block with different structures in the pre-training and fine-tuning phases,which can help achieve the goals of different training phases.To improve the robustness of the model,we add Gaussian noise to the training images as data augmentation.Our approach is effective in obtaining the best quantitative evaluation results using a small number of parameters and less training costs to improve the quality of handheld ultrasound devices.
文摘基于预训练和注意机制的意图分类和语义槽填充,提出一种结合双向长短时记忆(bidirectional long short-term memory,BiLSTM)、条件随机场(conditional random fields,CRF)和注意机制的双向编码(bidirectional encoder representations from transformers,BERT)具有双向编码表示和注意机制的联合模型。该模型无需过多依赖手工标签数据和领域特定的知识或资源,避免了目前普遍存在的弱泛化能力。在自主公交信息查询系统语料库上进行的试验表明,该模型意图分类的准确性和语义槽填充F1值分别达到98%和96.3%,均产生有效改进。
文摘人体目标检测对社会治理和城市安全具有很重要的现实意义,监控数据是数据安全的重要来源。小目标检测是目前受到广泛关注的安全检测问题中一项具有挑战性的任务,其检测对象为大型图像中少于20个像素的目标。小目标的特征难以表征,其中一个主要挑战是,用于预训练/共同训练检测器的数据集(如COCO)与用于微调检测器的数据集(如TinyPerson)之间存在尺度不匹配的情况,这给小目标检测器的性能带来了负面影响。为了解决这个问题,文中提出了一种优化策略,用于匹配不同数据集的尺度,称其为尺度分布搜索(Scale Distribution Search,SDS),同时平衡图片的信息收益(数据集之间的尺度相近)和信息损失(信噪比(SNR)的降低)。该策略使用高斯模型对数据集中目标的尺度分布进行建模,通过迭代的方式寻找最优分布参数;并对比数据集中目标的特征分布和检测器的性能,以找到最佳的尺度分布。通过SDS策略,主流目标检测方法在TinyPerson上实现了更好的性能,证明了SDS策略在提升预训练/共同训练效率上的有效性。
基金partially supported by the Natural Science Foundation of the Jiangsu Higher Education Institutions of China under Grant No.18kJB510010the Social Science Foundation of Jiangsu Province of China under Grant No.19TQD002the National Nature Science Foundation of China under Grant No.61976114.
文摘Graph neural networks(GNNs)have shown great power in learning on graphs.However,it is still a challenge for GNNs to model information faraway from the source node.The ability to preserve global information can enhance graph representation and hence improve classification precision.In the paper,we propose a new learning framework named G-GNN(Global information for GNN)to address the challenge.First,the global structure and global attribute features of each node are obtained via unsupervised pre-training,and those global features preserve the global information associated with the node.Then,using the pre-trained global features and the raw attributes of the graph,a set of parallel kernel GNNs is used to learn different aspects from these heterogeneous features.Any general GNN can be used as a kernal and easily obtain the ability of preserving global information,without having to alter their own algorithms.Extensive experiments have shown that state-of-the-art models,e.g.,GCN,GAT,Graphsage and APPNP,can achieve improvement with G-GNN on three standard evaluation datasets.Specially,we establish new benchmark precision records on Cora(84.31%)and Pubmed(80.95%)when learning on attributed graphs.
文摘近年来,网络安全威胁日益增多,数据驱动的安全智能分析成为网络安全领域研究的热点。特别是以知识图谱为代表的人工智能技术可为多源异构威胁情报数据中的复杂网络攻击检测和未知网络攻击检测提供支撑。网络安全实体识别是威胁情报知识图谱构建的基础。开放网络文本数据中的安全实体构成非常复杂,导致传统的深度学习方法难以准确识别。在BERT(pre-training of deep bidirectional transformers)预训练语言模型的基础上,提出一种基于残差空洞卷积神经网络和条件随机场的网络安全实体识别模型BERT-RDCNN-CRF。通过BERT模型训练字符级特征向量表示,结合残差卷积与空洞神经网络模型有效提取安全实体的重要特征,最后通过CRF获得每一个字符的BIO标注。在所构建的大规模网络安全实体标注数据集上的实验表明,所提方法取得了比LSTM-CRF模型、BiLSTM-CRF模型和传统的实体识别模型更好的效果。
文摘【目的】为了解决开源非结构化军事领域数据的命名实体识别问题。【方法】本文提出基于预训练模型(Bidirectional Encoder Representations from Transformers,BERT)的命名实体识别方法,首先基于预训练BERT模型生成自建开源军事语料的动态特征词向量的字符表示,然后基于双向长短期记忆网络(Bi-directional Long Short-Term Memory,BiLSTM)完成语义特征提取,最后利用条件随机场模型(Conditional Random Fields,CRF)选出最优标签序列完成实体识别的任务。【结果】模型在自建的开源军事数据集上的实验结果表明,相较于基于统计模型和神经网络的方法,本文提出的方法可使准确率提升8%,F值提高11%,召回率提高10%。【局限】由于现阶段在开源军事领域中公开标注数据集较为缺乏,所以未能在开源军事语料上训练BERT模型。【结论】但本文提出的基于预训练模型的开源军事命名实体识别方法,在一定程度上解决了边界划分问题,同时解决了在数据集不足的情况下实体识别任务表现不佳的问题。
