中北大学学报（自然科学版）2025，Vol.46Issue(5)：549-560,12.DOI:10.62756/jnuc.issn.1673-3193.2025.01.0006
基于膨胀卷积与图注意聚合的多模态医学图像融合
Multimodal Medical Image Fusion Based on Dilated Convolution and Graph Attention Aggregation
摘要
Abstract
Existing deep learning-based multimodal medical image fusion methods suffer from insufficient high-level feature extraction and easy loss of low-level features.To tackle these problems,this paper pro-posed a multimodal medical image fusion method based on dilated convolution and graph attention aggrega-tion.The method was comprised of three components:a dual-branch encoder,a fusion module,and a decoder.The dual-branch encoder consisted of a convolution-based low-level encoder and a graph-convolution-based high-level encoder.The convolution-based low-level encoder employed dilated convolu-tion to mitigate the loss of low-level features like texture details and provided initialized node features for the high-level encoder.The graph-convolution-based high-level encoder mainly adopted the graph atten-tion aggregation module to effectively capture high-level features such as deep semantics.The graph atten-tion aggregation module constructed a node adjacency matrix by integrating multi-head attention with edge encoding and then performed deep aggregation of nodes through graph convolution based on this adjacency matrix.The fusion module fused the extracted features,and the decoder reconstructed the fused image.The method was compared with six state-of-the-art image fusion methods on subjective vision and objec-tive evaluation metrics.The results show that this method improves 2.4%on EN compared to the IGNet method,3.53%and 5.06%on AG and MI compared to the DATFuse method,and 1.18%,6.24%,and 3%on SD,SF,and SCD compared to the SwinFusion method,respectively,while the fused image obtained by this method retains more texture detail information.The comprehensive experimental results demonstrate that this method achieves effective fusion of multimodal medical images,offering more reli-able image support for clinical diagnosis.关键词
多模态医学图像融合/双分支编码器/膨胀卷积/图卷积/多头注意Key words
multimodal medical image fusion/dual-branch encoder/dilated convolution/graph convolu-tion/multi-head attention分类
信息技术与安全科学引用本文复制引用
靳凯欣,王丽芳,郭威,韩强,郁晓庆..基于膨胀卷积与图注意聚合的多模态医学图像融合[J].中北大学学报（自然科学版）,2025,46(5):549-560,12.基金项目
山西省"1331工程"科技创新计划(20210222) （20210222）
山西省重点研发项目(202202010101008) （202202010101008）
山西省重点研发项目(202102010101011) （202102010101011）
山西省省筹资金资助回国留学人员科研项目(2024-118) （2024-118）
