为压缩网络剪枝过程中较大的搜索空间,从预训练深度神经网络中找到最佳的稀疏网络结构,本文提出一种基于遗传算法和知识蒸馏的分块压缩学习算法(CCLA).首先,在预定义压缩空间中将学生网络初始化为一个稀疏网络结构.然后,将教师网络和学...为压缩网络剪枝过程中较大的搜索空间,从预训练深度神经网络中找到最佳的稀疏网络结构,本文提出一种基于遗传算法和知识蒸馏的分块压缩学习算法(CCLA).首先,在预定义压缩空间中将学生网络初始化为一个稀疏网络结构.然后,将教师网络和学生网络按层划分多个块网络,在教师网络的监督下对学生网络进行逐块压缩.最后,使用遗传算法搜索学生网络中每个块网络的稀疏结构.在CIFAR-10数据集上对VGG-16网络和ResNet-110网络的实验结果表明,本文所提方法性能表现良好,例如,在CIFAR-10数据集上对VGG-16进行压缩,可压缩掉87.82%的参数和70.4%的浮点运算(Floating Point of operations,FLOPs),而精度损失仅为0.37%.展开更多
In academia and industries,graph neural networks(GNNs)have emerged as a powerful approach to graph data processing ranging from node classification and link prediction tasks to graph clustering tasks.GNN models are us...In academia and industries,graph neural networks(GNNs)have emerged as a powerful approach to graph data processing ranging from node classification and link prediction tasks to graph clustering tasks.GNN models are usually handcrafted.However,building handcrafted GNN models is difficult and requires expert experience because GNN model components are complex and sensitive to variations.The complexity of GNN model components has brought significant challenges to the existing efficiencies of GNNs.Hence,many studies have focused on building automated machine learning frameworks to search for the best GNN models for targeted tasks.In this work,we provide a comprehensive review of automatic GNN model building frameworks to summarize the status of the field to facilitate future progress.We categorize the components of automatic GNN model building frameworks into three dimensions according to the challenges of building them.After reviewing the representative works for each dimension,we discuss promising future research directions in this rapidly growing field.展开更多
以小世界模型为理论基础,以 Region 为基本逻辑管理单位,按用户需求和共享目的组织 Region。提出了基于 Region 的多层结构 Peer-to-Peer 网络模型和构造规则,给出了 Region 的划分策略和数学模型,证明了模型的正确和合理性;对模型中的...以小世界模型为理论基础,以 Region 为基本逻辑管理单位,按用户需求和共享目的组织 Region。提出了基于 Region 的多层结构 Peer-to-Peer 网络模型和构造规则,给出了 Region 的划分策略和数学模型,证明了模型的正确和合理性;对模型中的层和域、中心节点、普通节点和汇聚点进行了明确的定义,给出了节点加入、离开、中心节点选取策略和算法描述;使定位某种服务的工作量和查询范围从网络中的所有结点数降低到 Region 的节点数,有效地防止了恶意请求引发的洪,网络系统开销为常数。模拟分析表明,该模型可有效解决可扩展性、性能与效率不高问题,且网络规模越大,其综合性能的优越性越明显,因此,模型是合理有效的。展开更多
文摘为压缩网络剪枝过程中较大的搜索空间,从预训练深度神经网络中找到最佳的稀疏网络结构,本文提出一种基于遗传算法和知识蒸馏的分块压缩学习算法(CCLA).首先,在预定义压缩空间中将学生网络初始化为一个稀疏网络结构.然后,将教师网络和学生网络按层划分多个块网络,在教师网络的监督下对学生网络进行逐块压缩.最后,使用遗传算法搜索学生网络中每个块网络的稀疏结构.在CIFAR-10数据集上对VGG-16网络和ResNet-110网络的实验结果表明,本文所提方法性能表现良好,例如,在CIFAR-10数据集上对VGG-16进行压缩,可压缩掉87.82%的参数和70.4%的浮点运算(Floating Point of operations,FLOPs),而精度损失仅为0.37%.
基金supported by the National Natural Science Foundation of China(No.61873288)the CAAIHuawei Mind Spore Open Fund**。
文摘In academia and industries,graph neural networks(GNNs)have emerged as a powerful approach to graph data processing ranging from node classification and link prediction tasks to graph clustering tasks.GNN models are usually handcrafted.However,building handcrafted GNN models is difficult and requires expert experience because GNN model components are complex and sensitive to variations.The complexity of GNN model components has brought significant challenges to the existing efficiencies of GNNs.Hence,many studies have focused on building automated machine learning frameworks to search for the best GNN models for targeted tasks.In this work,we provide a comprehensive review of automatic GNN model building frameworks to summarize the status of the field to facilitate future progress.We categorize the components of automatic GNN model building frameworks into three dimensions according to the challenges of building them.After reviewing the representative works for each dimension,we discuss promising future research directions in this rapidly growing field.
文摘以小世界模型为理论基础,以 Region 为基本逻辑管理单位,按用户需求和共享目的组织 Region。提出了基于 Region 的多层结构 Peer-to-Peer 网络模型和构造规则,给出了 Region 的划分策略和数学模型,证明了模型的正确和合理性;对模型中的层和域、中心节点、普通节点和汇聚点进行了明确的定义,给出了节点加入、离开、中心节点选取策略和算法描述;使定位某种服务的工作量和查询范围从网络中的所有结点数降低到 Region 的节点数,有效地防止了恶意请求引发的洪,网络系统开销为常数。模拟分析表明,该模型可有效解决可扩展性、性能与效率不高问题,且网络规模越大,其综合性能的优越性越明显,因此,模型是合理有效的。
