四川轻化工大学学报（自然科学版）2024，Vol.37Issue(4)：36-46,11.DOI:10.11863/j.suse.2024.04.05
基于文本特征融合的双流生成对抗修复网络
Dual-stream Generative Adversarial Repair Network Based on Text Feature Fusion
摘要
Abstract
In order to solve the problems of insufficient feature mining and incomplete semantic expression in deep learning technology,and to eliminate the phenomenon of artifacts or fuzzy texture in repaired images,a dual-stream generative adversarial restoration network with context feature fusion is constructed in the present paper,which is used to realize reconstruction,perception and style loss compensation,so as to achieve global consistency of repaired images.Integrated with the attention mechanism,the U-Net is used as the backbone network to fully extract the image structure and texture features in this network.The upper and lower text feature fusion network is used to fully excavate the high-level semantic and contextual relationship of the feature information,to realize the filling and fine restoration of the structure and texture features of the empty region.The structure and texture dual-stream discriminator is used to estimate the features of texture and structure and distinguish the real image from the generated image.A semantic-based joint loss function is used to enhance the semantic authenticity of repaired images.The CTSDG algorithm with the best performance among the comparison algorithms is compared based on CelebA and Places2 datasets.The values of PSNR and SSIM on CelebA are increased by 2.74 dB and 5.80%respectively,while the value of FID is decreased by 4.02.Similarly,the values of PSNR and SSIM on Place2 increases by 4.15 dB and 3.33%respectively,while the value of FID decreases of 2.33.Therefore,the improved image restoration method shows better objective evaluation indexes,which can repair the structure and texture information of damaged images more effectively,and makes the performance of image restoration better.关键词
注意力机制/双流结构/生成对抗网络/双流鉴别器/联合损失函数Key words
attention mechanisms/dual-stream structures/generative adversarial networks/dual-stream discriminator/joint loss function分类
信息技术与安全科学引用本文复制引用
刘婷婷,陈明举,李兰..基于文本特征融合的双流生成对抗修复网络[J].四川轻化工大学学报（自然科学版）,2024,37(4):36-46,11.基金项目
四川省科技成果转移转化示范项目(2022ZHCG0035) （2022ZHCG0035）
海南省Internet信息检索重点实验室开放基金项目(2022KY03) （2022KY03）
四川省机器人与智能系统国际联合研究中心开放研究课题(JQZN2022-005) （JQZN2022-005）
四川轻化工大学研究生创新基金项目(Y2022130) （Y2022130）
