摘要
变分自编码器(VAE)作为一个功能强大的文本生成模型受到越来越多的关注。然而,变分自编码器在优化过程中容易出现后验崩溃,即忽略潜在变量,退化为一个自编码器。针对这个问题,该文提出一种新的变分自编码器模型,通过层次化编码和状态正则方法,可以有效缓解后验崩溃,且相较于基线模型具有更优的文本生成质量。在此基础上,基于纳米级忆阻器,将提出的变分自编码器模型与忆阻循环神经网络(RNN)结合,设计一种基于忆阻循环神经网络的硬件实现方案,即层次化变分自编码忆组神经网络(HVAE-MNN),探讨模型的硬件加速。计算机仿真实验和结果分析验证了该文模型的有效性与优越性。
As a powerful text generation model, the Variational AutoEncoder(VAE) has attracted more and more attention. However, in the process of optimization, the variational auto-encoder tends to ignore the potential variables and degenerates into an auto-encoder, called a posteriori collapse. A new variational autoencoder model is proposed in this paper, called Hierarchical Status Regularisation Variational AutoEncoder(HSR-VAE), which can effectively alleviate the problem of posterior collapse through hierarchical coding and state regularization and has better model performance than the baseline model. On this basis, based on the nanometer memristor, the model is combined with the memristor Recurrent Neural Network(RNN). A hardware implementation scheme based on a memristor recurrent neural network is proposed to realize the hardware acceleration of the model, which called Hierarchical Variational AutoEncoder Memristor Neural Networks(HVAE-MHN). Computer simulation experiments and result analysis verify the validity and superiority of the proposed model.
作者
胡小方
杨涛
HU Xiaofang;YANG Tao(College of Artificial Intelligence,Southwest University,Chongqing 400715,China;Brain-inspired Computing&Intelligent Control of Chongqing Key Laboratory,Chongqing 400715,China)
出处
《电子与信息学报》
EI
CSCD
北大核心
2023年第2期689-697,共9页
Journal of Electronics & Information Technology
基金
国家自然科学基金(61976246)
重庆市自然科学基金(cstc2020jcyj-msxm X0385)。
关键词
变分自编码器
忆阻器
忆阻循环网络
文本生成
Variational AutoEncoder(VAE)
Memristor
Memristor recurrent network
Text generation
