空军工程大学学报2024，Vol.25Issue(4)：118-127,10.DOI:10.3969/j.issn.2097-1915.2024.04.016
结合语义分割图的注意力机制文本生成图像
A Semantic Segmentation Graph in Combination with Attention Mechanism Text Generation Images
摘要
Abstract
Aimed at the problems that generative adversarial network is incomplete in structure,unreal in content and poor in quality of images generated,an attention mechanism text-to-image generation model combined with semantic segmentation graph(SSA-GAN)is proposed.First,taking global sentence vec-tors as input conditions,a simple and effective deep fusion module is utilized for fully fusing text informa-tion while generating images are generating simultaneously.Second,the semantically segmented images are combined to extract their edge profile features to provide additional generative and constraint conditions for the model,and the attention mechanism is used to provide fine-grained word-level information for the model to enrich the details of the generated images.Finally,a multimodal similarity computation model is used to compute fine-grained image-text matching loss to further train the generator.The model is tested and validated by CUB-200 and Oxford-102 Flowers datasets,and the results show that the proposed model(SSA-GAN)improves the quality of the final generated images.Compared to the models such as Stack-GAN,AttnGAN,DF-GAN,and RAT-GAN,the IS increases in metrics values by 13.7%and 43.2%,re-spectively.And the FID in metric values is reduced to 34.7%and 74.9%,respectively.关键词
文本生成图像/语义分割图像/生成对抗网络/注意力机制/仿射变换Key words
text generates images/semantic segmentation image/attention mechanism/generate adversarial network/affine transformation分类
计算机与自动化引用本文复制引用
梁成名,李云红,李丽敏,苏雪平,朱绵云,朱耀麟..结合语义分割图的注意力机制文本生成图像[J].空军工程大学学报,2024,25(4):118-127,10.基金项目
国家自然科学基金(62203344) （62203344）
陕西省自然科学基础研究重点项目(2022JZ-35) （2022JZ-35）
陕西高校青年创新团队项目 （）
