摘要
基于图像合成蒙版策略的虚拟试衣技术在扭曲服装和人体融合时能较好地保留服装细节。然而由于在试衣过程中人体和服装的位置和结构难以对齐,试衣结果容易产生严重的遮挡,影响视觉效果。为解决试衣过程中的遮挡问题,提出了一种基于U-Net的生成器。该生成器在U-Net解码器上添加级联的空间和通道注意力模块,从而实现了着装人体的局部特征和扭曲服装的和全局特征的跨域融合。形式上,首先采用卷积网络预测薄板样条插值(TPS)变换的方法将服装根据目标人体姿态进行扭曲;然后,将着装人体解析信息和扭曲服装输入到提出的生成器中,并获取对应服装区域的掩码图像以渲染中间结果;最后,采用掩码合成的策略来通过掩码处理将扭曲服装与中间结果合成得到最终的试衣结果。实验结果表明,所提方法不仅可以减少遮挡,而且增强了图像细节,相较于CP-VTON方法,产生的图像的平均峰值信噪比(PSNR)提高了10.47%,平均FID减小了47.28%,平均结构相似性(SSIM)提高了4.16%。
The virtual try-on technologies based on image synthesis mask strategy can better retain details of the clothing when the warped clothing is fused with the human body.However,because the position and structure of the human body and the clothing are difficult to align during the try-on process,the try-on result is likely to produce severe occlusion,affecting visual effect.In order to solve the occlusion in the try-on process,a U-Net based generator was proposed.In the generator,a cascaded spatial attention module and a channel attention module were added to the U-Net decoder,thereby achieving the cross-domain fusion between local features of warped clothes and global features of the human body.Formally,first,by predicting the Thin Plate Spline(TPS)conversion using the convolutional network,the clothing was distorted according to the target human body pose.Then,the dressed-on person representation information and the warped clothing were input into the proposed generator,and the mask image of the corresponding clothing area was obtained to render the intermediate result.Finally,the strategy of mask synthesis was used to synthesize the warped clothing with the intermediate result through mask processing to obtain the final try-on result.Experimental results show that the proposed method can not only reduce occlusion,but also enhance image details.Compared with Characteristic-Preserving Virtual Try-On Network(CP-VTON)method,the proposed method has the generated image with the average Peak Signal-to-Noise Ratio(PSNR)increased by 10.47%,the average Fréchet Inception Distance(FID)decreased by 47.28%,and the average Structural SIMilarity(SSIM)increased by 4.16%.
作者
胡新荣
张君宇
彭涛
刘军平
何儒汉
何凯
HU Xinrong;ZHANG Junyu;PENG Tao;LIU Junping;HE Ruhan;HE Kai(Engineering Research Center of Hubei Province in Clothing Informationization(Wuhan Textile University),Wuhan Hubei 430200,China;School of Computer Science and Artificial Intelligence,Wuhan Textile University,Wuhan Hubei 430200,China)
出处
《计算机应用》
CSCD
北大核心
2022年第4期1269-1274,共6页
journal of Computer Applications
基金
湖北省高等学校优秀中青年科技创新团队计划项目(T201807)。
关键词
虚拟试衣
注意力机制
特征融合
遮挡处理
级联
跨域特征融合
virtual try-on
attention mechanism
feature fusion
occlusion processing
cascaded
cross-domain feature fusion