人工神经网络的自适应结构学习(AdaNet)是基于Boosting集成学习的神经结构搜索框架,可通过集成子网创建高质量的模型。现有的AdaNet所产生的子网之间的差异性不显著,因而限制了集成学习中泛化误差的降低。在AdaNet设置子网网络权重和集...人工神经网络的自适应结构学习(AdaNet)是基于Boosting集成学习的神经结构搜索框架,可通过集成子网创建高质量的模型。现有的AdaNet所产生的子网之间的差异性不显著,因而限制了集成学习中泛化误差的降低。在AdaNet设置子网网络权重和集成子网的两个步骤中,使用Adagrad、RMSProp、Adam、RAdam等自适应学习率方法来改进现有AdaNet中的优化算法。改进后的优化算法能够为不同维度参数提供不同程度的学习率缩放,得到更分散的权重分布,以增加AdaNet产生子网的多样性,从而降低集成学习的泛化误差。实验结果表明,在MNIST(Mixed National Institute of Standards and Technology database)、Fashion-MNIST、带高斯噪声的Fashion-MNIST这三个数据集上,改进后的优化算法能提升AdaNet的搜索速度,而且该方法产生的更加多样性的子网能提升集成模型的性能。在F1值这一评估模型性能的指标上,改进后的方法相较于原方法,在三种数据集上的最大提升幅度分别为0.28%、1.05%和1.10%。展开更多
Neural architecture search(NAS)has become increasingly popular in the deep learning community recently,mainly because it can provide an opportunity to allow interested users without rich expertise to benefit from the ...Neural architecture search(NAS)has become increasingly popular in the deep learning community recently,mainly because it can provide an opportunity to allow interested users without rich expertise to benefit from the success of deep neural networks(DNNs).However,NAS is still laborious and time-consuming because a large number of performance estimations are required during the search process of NAS,and training DNNs is computationally intensive.To solve this major limitation of NAS,improving the computational efficiency is essential in the design of NAS.However,a systematic overview of computationally efficient NAS(CE-NAS)methods still lacks.To fill this gap,we provide a comprehensive survey of the state-of-the-art on CE-NAS by categorizing the existing work into proxy-based and surrogate-assisted NAS methods,together with a thorough discussion of their design principles and a quantitative comparison of their performances and computational complexities.The remaining challenges and open research questions are also discussed,and promising research topics in this emerging field are suggested.展开更多
In most popular public accessible cryptocurrency systems,the mining pool plays a key role because mining cryptocurrency with the mining pool turns the non-profitable situation into profitable for individual miners.In ...In most popular public accessible cryptocurrency systems,the mining pool plays a key role because mining cryptocurrency with the mining pool turns the non-profitable situation into profitable for individual miners.In many recent novel blockchain consensuses,the deep learning training procedure becomes the task for miners to prove their workload.Thus,the computation power of miners will not purely be spent on the hash puzzle.In this way,the hardware and energy will support the blockchain service and deep learning training simultaneously.While the incentive of miners is to earn tokens,individual miners are motivated to join mining pools to become more competitive.In this paper,we are the first to demonstrate a mining pool solution for novel consensuses based on deep learning.The mining pool manager partitions the full searching space into subspaces,and all miners are scheduled to collaborate on the Neural architecture search(NAS)tasks in the assigned subspace.Experiments demonstrate that the performance of this type of mining pool is more competitive than that of an individual miner.Due to the uncertainty of miners'behaviors,the mining pool manager checks the standard deviation of the performance of high reward miners and prepares backup miners to ensure completion of the tasks of high reward miners.展开更多
文摘人工神经网络的自适应结构学习(AdaNet)是基于Boosting集成学习的神经结构搜索框架,可通过集成子网创建高质量的模型。现有的AdaNet所产生的子网之间的差异性不显著,因而限制了集成学习中泛化误差的降低。在AdaNet设置子网网络权重和集成子网的两个步骤中,使用Adagrad、RMSProp、Adam、RAdam等自适应学习率方法来改进现有AdaNet中的优化算法。改进后的优化算法能够为不同维度参数提供不同程度的学习率缩放,得到更分散的权重分布,以增加AdaNet产生子网的多样性,从而降低集成学习的泛化误差。实验结果表明,在MNIST(Mixed National Institute of Standards and Technology database)、Fashion-MNIST、带高斯噪声的Fashion-MNIST这三个数据集上,改进后的优化算法能提升AdaNet的搜索速度,而且该方法产生的更加多样性的子网能提升集成模型的性能。在F1值这一评估模型性能的指标上,改进后的方法相较于原方法,在三种数据集上的最大提升幅度分别为0.28%、1.05%和1.10%。
基金This work was supported by a Ulucu PhD studentshipY.Jin is funded by an Alexander von Humboldt Professorship for Artificial Intelligence endowed by the German Federal Ministry of Education and Research.
文摘Neural architecture search(NAS)has become increasingly popular in the deep learning community recently,mainly because it can provide an opportunity to allow interested users without rich expertise to benefit from the success of deep neural networks(DNNs).However,NAS is still laborious and time-consuming because a large number of performance estimations are required during the search process of NAS,and training DNNs is computationally intensive.To solve this major limitation of NAS,improving the computational efficiency is essential in the design of NAS.However,a systematic overview of computationally efficient NAS(CE-NAS)methods still lacks.To fill this gap,we provide a comprehensive survey of the state-of-the-art on CE-NAS by categorizing the existing work into proxy-based and surrogate-assisted NAS methods,together with a thorough discussion of their design principles and a quantitative comparison of their performances and computational complexities.The remaining challenges and open research questions are also discussed,and promising research topics in this emerging field are suggested.
文摘In most popular public accessible cryptocurrency systems,the mining pool plays a key role because mining cryptocurrency with the mining pool turns the non-profitable situation into profitable for individual miners.In many recent novel blockchain consensuses,the deep learning training procedure becomes the task for miners to prove their workload.Thus,the computation power of miners will not purely be spent on the hash puzzle.In this way,the hardware and energy will support the blockchain service and deep learning training simultaneously.While the incentive of miners is to earn tokens,individual miners are motivated to join mining pools to become more competitive.In this paper,we are the first to demonstrate a mining pool solution for novel consensuses based on deep learning.The mining pool manager partitions the full searching space into subspaces,and all miners are scheduled to collaborate on the Neural architecture search(NAS)tasks in the assigned subspace.Experiments demonstrate that the performance of this type of mining pool is more competitive than that of an individual miner.Due to the uncertainty of miners'behaviors,the mining pool manager checks the standard deviation of the performance of high reward miners and prepares backup miners to ensure completion of the tasks of high reward miners.
