基于多模态融合和自适应剪枝Transformer的脑肿瘤图像分割算法OACHSSCDCSTPCD
Brain Tumor Segmentation Algorithm Based on Multi-modal Fusion and Adaptive Pruning Transformer
脑肿瘤是目前世界上最致命的肿瘤之一,所以脑肿瘤图像的自动分割在临床诊疗中变得日益重要.近年来,基于CNN和Transformer的脑肿瘤分割方法在医学图像分割领域取得了令人欣喜的成就.然而,大多数方法没有充分利用脑肿瘤多模态间的互补性和差异性,并且模型中的Transformer在捕获远程依赖性的同时,忽略了其较大的计算复杂性、冗余依赖性等问题.针对此问题,提出一种基于多模态融合和自适应剪枝Transformer的脑肿瘤图像分割方法(MF-MAPT Swin UNETR),其中多模态融合模块可以充分学习性质相近的模态间信息和不同模态不同尺度的特征变化,为后续分割提供了充分的准备;基于多模态的自适应剪枝Transformer可以降低计算复杂度,对提升性能有一定的帮助,将MF-MAPT Swin UNETR模型在两个公共数据集上进行了实验验证,结果表明,该模型较最先进的方法整体具有突出的分割性能.
Brain tumors are one of the most lethal tumors in the world,so automatic segmentation of brain tumor images is becoming increasingly important in clinical diagnosis and treatment.In recent years,brain tumor segmentation methods based on CNN and Transformer have achieved gratifying achievements in the field of medical image segmentation.However,in most methods the complementarity and difference between brain tumor multimodalities are not fully exploited,and in methods with Transformer where long-range dependence is captured,the large computational complexity and redundant dependencies are unbearable.To solve the two problems,a brain tumor image segmentation method(MF-MAPT Swin UNETR)based on multi-modal fusion and adaptive pruning Transformer is proposed.The multi-modal fusion module can fully learn the charac-teristics between modalities with similar properties and feature changes in different modes and scales provide suf-ficient preparation for subsequent segmentation;the multi-modal adaptive pruning Transformer can reduce com-putational complexity and help improve performance.The model is experimentally verified on two public datas-ets,and it is shown by the experimental results that the proposed MF-MAPT Swin UNETR model exhibits outstanding segmentation performance overall compared with state-of-the-art methods.
姚宗亮;黄荣;董爱华;韩芳;王青云
东华大学 信息科学与技术学院,上海 201620东华大学 信息科学与技术学院,上海 201620||东华大学 数字化纺织服装技术教育部工程研究中心,上海 201620宁夏大学 数学统计学院,宁夏 银川 750021||北京航空航天大学 动力学与控制系,北京 100191
计算机与自动化
脑肿瘤分割Transformer模态交叉连接多尺度特征融合token融合自适应剪枝
brain tumor segmentationTransformermodal cross-connectionmulti-scale feature fusiontoken fusionadaptive pruning
《宁夏大学学报(自然科学版)》 2024 (001)
16-24 / 9
国家自然科学基金资助项目(12272092,62001099)
评论