卷积神经网络性能的快速提升是以不断堆叠的网络层数以及成倍增长的参数量和存储空间为代价,这不仅会使模型在训练过程中出现过拟合等问题,也不利于模型在资源受限的嵌入式设备上运行,因而提出模型压缩技术来解决上述问题,主要对模型压...卷积神经网络性能的快速提升是以不断堆叠的网络层数以及成倍增长的参数量和存储空间为代价,这不仅会使模型在训练过程中出现过拟合等问题,也不利于模型在资源受限的嵌入式设备上运行,因而提出模型压缩技术来解决上述问题,主要对模型压缩技术中的特征蒸馏算法进行了研究。针对特征蒸馏中利用教师网络特征图指导学生网络并不能很好地锻炼学生网络特征拟合能力的问题,提出基于特征分布蒸馏算法。该算法利用条件互信息的概念构建模型特征空间的概率分布,并引入最大平均差异(maximum mean discrepancy,MMD)设计损失函数以最小化教师网络和学生网络特征分布间的距离。在知识蒸馏的基础上利用toeplitz矩阵对学生网络进行权重共享操作,进一步节省了模型的存储空间。为验证在特征分布蒸馏算法训练下学生网络的特征拟合能力,在图像分类、目标检测和语义分割三种图像处理任务上进行了实验验证,实验表明所提算法在以上三种学习任务中的表现均优于对比算法且实现了不同网络架构间的蒸馏。展开更多
The quality of the radiation dose depends upon the gamma count rate of the radionuclide used. Any reduction in error in the count rate is reflected in the reduction in error in the activity and consequently on the qua...The quality of the radiation dose depends upon the gamma count rate of the radionuclide used. Any reduction in error in the count rate is reflected in the reduction in error in the activity and consequently on the quality of dose. All the efforts so far have been directed only to minimize the random errors in count rate by repetition. In the absence of probability distribution for the systematic errors, we propose to minimize these errors by estimating the upper and lower limits by the technique of determinant in equalities developed by us. Using the algorithm we have developed based on the tech- nique of determinant inequalities and the concept of maximization of mutual information (MI), we show how to process element by element of the covariance matrix to minimize the correlated systematic errors in the count rate of 113 mIn. The element wise processing of covariance matrix is so unique by our technique that it gives experimentalists enough maneuverability to mitigate different factors causing systematic errors in the count rate and consequently the activity of 113 mIn.展开更多
We have applied concepts from information theory for a comparative analysis of donor （gt） and acceptor （ag） splice site regions in the genes of five different organisms by calculating their mutual information cont...We have applied concepts from information theory for a comparative analysis of donor （gt） and acceptor （ag） splice site regions in the genes of five different organisms by calculating their mutual information content （relative entropy） over a selected block of nucleotides. A similar pattern that the information content decreases as the block size increases was observed for both regions in all the organisms studied. This result suggests that the information required for splicing might be contained in the consensus of -6-8 nt at both regions. We assume from our study that even though the nucleotides are showing some degrees of conservation in the flanking regions of the splice sites, certain level of variability is still tolerated, which leads the splicing process to occur normally even if the extent of base pairing is not fully satisfied. We also suggest that this variability can be compensated by recognizing different splice sites with different spliceosomal factors.展开更多
文摘卷积神经网络性能的快速提升是以不断堆叠的网络层数以及成倍增长的参数量和存储空间为代价,这不仅会使模型在训练过程中出现过拟合等问题,也不利于模型在资源受限的嵌入式设备上运行,因而提出模型压缩技术来解决上述问题,主要对模型压缩技术中的特征蒸馏算法进行了研究。针对特征蒸馏中利用教师网络特征图指导学生网络并不能很好地锻炼学生网络特征拟合能力的问题,提出基于特征分布蒸馏算法。该算法利用条件互信息的概念构建模型特征空间的概率分布,并引入最大平均差异(maximum mean discrepancy,MMD)设计损失函数以最小化教师网络和学生网络特征分布间的距离。在知识蒸馏的基础上利用toeplitz矩阵对学生网络进行权重共享操作,进一步节省了模型的存储空间。为验证在特征分布蒸馏算法训练下学生网络的特征拟合能力,在图像分类、目标检测和语义分割三种图像处理任务上进行了实验验证,实验表明所提算法在以上三种学习任务中的表现均优于对比算法且实现了不同网络架构间的蒸馏。
文摘The quality of the radiation dose depends upon the gamma count rate of the radionuclide used. Any reduction in error in the count rate is reflected in the reduction in error in the activity and consequently on the quality of dose. All the efforts so far have been directed only to minimize the random errors in count rate by repetition. In the absence of probability distribution for the systematic errors, we propose to minimize these errors by estimating the upper and lower limits by the technique of determinant in equalities developed by us. Using the algorithm we have developed based on the tech- nique of determinant inequalities and the concept of maximization of mutual information (MI), we show how to process element by element of the covariance matrix to minimize the correlated systematic errors in the count rate of 113 mIn. The element wise processing of covariance matrix is so unique by our technique that it gives experimentalists enough maneuverability to mitigate different factors causing systematic errors in the count rate and consequently the activity of 113 mIn.
文摘We have applied concepts from information theory for a comparative analysis of donor （gt） and acceptor （ag） splice site regions in the genes of five different organisms by calculating their mutual information content （relative entropy） over a selected block of nucleotides. A similar pattern that the information content decreases as the block size increases was observed for both regions in all the organisms studied. This result suggests that the information required for splicing might be contained in the consensus of -6-8 nt at both regions. We assume from our study that even though the nucleotides are showing some degrees of conservation in the flanking regions of the splice sites, certain level of variability is still tolerated, which leads the splicing process to occur normally even if the extent of base pairing is not fully satisfied. We also suggest that this variability can be compensated by recognizing different splice sites with different spliceosomal factors.
