液晶与显示2025，Vol.40Issue(8)：1189-1201,13.DOI:10.37188/CJLCD.2025-0087
基于Restormer与双重注意力机制的非对称医学图像融合模型
Asymmetric medical image fusion model based on Restormer and dual attention mechanism
摘要
Abstract
There are differences in spatial information distribution among different medical imaging models,which is not conducive to the effective alignment of the depth feature space,resulting in the loss of shallow information in a specific area of the fusion image or excessive dependence on the information of a certain mode.To solve these problems,an asymmetric medical image fusion model based on Restormer and dual attention mechanism was proposed.Firstly,Restormer module is used to dig deep features of different modal images,and dual attention mechanism is introduced to extract global and local features of different modal images.Secondly,an asymmetric feature fusion strategy is designed,in which an independent feature encoder is designed for each mode and the extracted features are fused.Finally,the fused features are generated by the decoder.This model adopts two stages of training,the first stage mainly extracts global and local features from different modal images,and attempts to reconstruct the original image to calculate the loss;the second stage continues to extract deep features and generate fusion images.Compared with the seven mainstream image fusion models,the seven evaluation indicators,standard deviation,spatial frequency,visual information fidelity,spectral relevance,mutual information,average gradient,and Q index used to evaluate hybrid fusion have an average increase of 12.63%,28.30%,31.37%,27.40%,19.01%,37.36%,32.44%,respectively.The fusion strategy of this model can not only efficiently integrate the coding features from different modes,but also complete the integration of complementary information and the interaction of global information without manually designing fusion rules,and can better integrate images from different modes.关键词
医学图像融合/双重注意力机制/非对称融合/两阶段训练Key words
medical image fusion/dual attention mechanism/asymmetric fusion/two-stage training分类
信息技术与安全科学引用本文复制引用
孔韦韦,李泽江,何磊磊,杜玉胜..基于Restormer与双重注意力机制的非对称医学图像融合模型[J].液晶与显示,2025,40(8):1189-1201,13.基金项目
国家自然科学基金(No.62471389) （No.62471389）
陕西省科技厅面上项目(No.2022JM-369)Supported by National Natural Science Foundation of China(No.62471389) （No.2022JM-369）
Project of Science and Tech-nology Department of Shaanxi Province(No.2022JM-369) （No.2022JM-369）
