摘要
该文提出了一种改进条件生成对抗网络的文本生成图像模型(TxtGAN),使用一对生成器和判别器的单阶段生成方式生成高分辨率图像,避免因训练多个GAN消耗大量计算资源.生成器网络由一系列生成模块(RUPBlock)组成,每个模块中应用条件批量归一化方法,在实现图像生成的同时充分融合了文本信息与图像特征,较好地保留了文本信息.另外,将文本词信息引入判别器中,得到的对抗性损失作为“信号”引导生成器高效训练.在CUB鸟类数据集和COCO数据集上的实验结果表明,TxtGAN模型优于基线方法,同样可以生成较为真实的高分辨率的图像,并且较好地解决了多阶段训练的缺陷.
To generate high-resolution images, traditional text-to-image methods generally use multi-stage generation architecture, but there are problems such as high computational cost and loss of text information during generation. Therefore, a text-to-image model(TxtGAN) with improved conditional generative adversarial network is proposed to generate high-resolution images, which uses a single-stage generation of a pair of generators and discriminators to avoid consuming large computational resources due to training multiple GANs. The generator network consists of a series of generation blocks(RUPBlocks),and the conditional batch normalization method is applied in each block to fully integrate textual information with image features while achieving image generation, which better preserves textual information. In addition, textual word information is introduced into the discriminator, and the resulting adversarial loss is used as a “signal” to guide the efficient training of the generator. The experimental results on CUB bird dataset and COCO dataset show that the TxtGAN model outperforms the baseline method, generates more realistic and high-resolution images, and solves the shortcomings of multi-stage training better.
作者
侯丽君
倪建成
张素素
HOU Lijun;NI Jiancheng;ZHANG Susu(School of Cyber Science and Engineering,Qufu Normal University,273165,Qufu,Shandong,PRC)
出处
《曲阜师范大学学报（自然科学版）》
CAS
2022年第2期63-70,共8页
Journal of Qufu Normal University(Natural Science)
基金
国家自然科学基金青年项目(61601261)
山东省研究生教育质量提升计划项目(SDYY17136)。
关键词
生成对抗网络
残差网络
文本生成图像
条件融合
generative adversarial networks
residual network
text to image
condition fusion
