摘要
深度域适应作为迁移学习最常见的问题之一,已经在许多机器学习应用中获得了优异的性能。然而,现有的深度域适应方法在减小域偏差时单一适配完全连接层,忽视了卷积层的空间信息和语义上下文信息,造成在知识迁移过程中丢失重要信息。为此,文中将基于实例的域适应与基于特征的域适应相结合,提出了基于PE散度实例过滤的深度域适应方法(Domain Adaptation Based on PE Divergence Instance Filtering,DAPEIF)。其基本思想是首先利用PE散度计算源域样本的相对权值,删除易造成负迁移的源域样本,选择相对权值较高的训练数据作为新的源域样本,从而降低源域与目标域之间的差异性;然后基于AlexNet模型,使用最大均值差异(Maximum Mean Discrepancy,MMD)准则,将其作为正则化项纳入神经网络的学习中。与以往只关注完全连接层的域适应方法不同,文中联合匹配卷积层和完全连接层的边缘概率分布以解决欠适配问题,同时引入权值正则项,通过梯度下降法学习网络参数,进一步提高了域适应过程中模型的泛化性能。所提算法能同时对神经网络的卷积层和完全连接层的参数赋予领域自适应能力,在域适应公开数据集Office-Caltech上的实验结果表明,域适应对卷积层是有效的,更重要的是卷积层的自适应可以进一步提高传统域适应的性能,与主流的域适应算法DDC和DAN相比,所提算法在精度上有所提高,平均精度达到84.46%。
Deep domain adaptation,one of the most common problems in transfer learning,has achieved excellent performance in many machine learning applications.However,the existing deep domain adaptation just adapts the fully connected layer while reduce domain shift,which ignores the spatial information and semantic context information of the convolutional layer,resulting in the loss of important information during the process of knowledge transfer.To this end,this paper combines instance-based domain adaptation with feature-based domain adaptation and proposes a deep domain adaptation algorithm based on PE divergence instance filtering(DAPEIF).Firstly,the relative importance weight of the source samples is calculated by PE divergence,and the source samples that are likely to cause negative transfer are deleted.Then the maximum mean discrepancy metric is included as a regularization term in the training of the neural network based on AlexNet,and the marginal probability distribution of the convolutional layer and the fully connected layer are jointly adapted to reduce the difference between the new source domain and the target domain.At the same time,the weight regularization term is introduced,and the network parameters are learned by the gradient descent to further improve the generalization performance of the model in the process of domain adaptation.The proposed algorithm can simultaneously assign domain adaptative ability to the parameters of the convolutional layer and the fully connected layer of the neural network.The experimental results on the public dataset Office-Caltech in domain adaptation demonstrate that compared with the state of the art domain adaptation algorithms,the proposed algorithm has better performance in accuracy and the average accuracy reaches 84.46%.
作者
袁晨晖
程春玲
YUAN Chen-hui;CHENG Chun-ling(College of Computer Science,Nanjing University of Posts and Telecommunications,Nanjing 210023,China)
出处
《计算机科学》
CSCD
北大核心
2020年第8期151-156,共6页
Computer Science
基金
国家自然科学基金项目(61672297)。
关键词
迁移学习
深度域适应
域偏差
PE散度
最大均值差异
Transfer learning
Domain adaptation
Domain shift
Pearson divergence
Maximum mean discrepancy
