西南交通大学学报2024，Vol.59Issue(5)：1204-1214,11.DOI:10.3969/j.issn.0258-2724.20230529
基于注意力机制与光照感知网络的红外与可见光图像融合
Infrared and Visible Image Fusion Based on Attention Mechanism and Illumination-Aware Network
摘要
Abstract
Some image fusion methods do not fully consider the illumination conditions in the image environment,resulting in insufficient brightness of infrared targets and overall low brightness of the image in the fused image,thereby affecting the clarity of texture details.To address these issues,an infrared and visible image fusion algorithm based on attention mechanism and illumination-aware network was proposed.Firstly,before training the fusion network,the illumination-aware network was used to calculate the probability that the current scene was daytime or nighttime and apply it to the loss function of the fusion network,so as to guide the training of the fusion network.Then,in the feature extraction part of the network,spatial attention mechanism and depthwise separable convolution were used to extract features from the source image.After obtaining spatial salient information,it was input into a convolutional neural network(CNN)to extract deep features.Finally,the deep feature information was concatenated for image reconstruction to obtain the final fused image.The experimental results show that the method proposed in this paper improves mutual information(MI),visual fidelity(VIF),average gradient(AG),fusion quality(Qabf),and spatial frequency(SF)by an average of 39.33%,11.29%,26.27%,47.11%,and 39.01%,respectively.At the same time,it can effectively preserve the brightness of infrared targets in the fused images,including rich texture detail information.关键词
图像融合/注意力机制/卷积神经网络/红外特征提取/深度学习Key words
Image fusion/attention mechanism/convolutional neural network/infrared feature extraction/deep learning分类
信息技术与安全科学引用本文复制引用
杨艳春,闫岩,王可..基于注意力机制与光照感知网络的红外与可见光图像融合[J].西南交通大学学报,2024,59(5):1204-1214,11.基金项目
长江学者和创新团队发展计划(IRT_16R36) （IRT_16R36）
国家自然科学基金项目(62067006) （62067006）
甘肃省科技计划(18JR3RA104) （18JR3RA104）
甘肃省高等学校产业支撑计划(2020C-19) （2020C-19）
甘肃省教育厅青年博士基金项目(2022QB-067) （2022QB-067）
甘肃省自然科学基金项目(23JRRA847,21JR7RA300) （23JRRA847,21JR7RA300）
