计算机工程2025，Vol.51Issue(12)：304-310,7.DOI:10.19678/j.issn.1000-3428.0069544
基于多模态融合的宫颈上皮内瘤变辅助诊断
Assisted Diagnosis of Cervical Intraepithelial Neoplasia Based on Multimodal Fusion
摘要
Abstract
In recent years,deep learning has made significant progress in the field of medical image processing.Most existing methods use image classification to assist physicians in pathological diagnosis,which may suffer from a lack of interpretability.To address this problem,this study proposes a multimodal feature fusion-based classification model for Cervical Intraepithelial Neoplasia(CIN).In this multimodal classification model,a patient's pathology image is used as the image modality data,whereas the corresponding pathology report description is used as the text modality data.Most current multimodal models use a simple splicing method to combine the features of each modality when dealing with different modal information.However,this method often ignores the connection between cross-modal information.To facilitate the interaction between cross-modal information,this study adopts an improved nonlocal attention mechanism to enhance the model's ability to understand the data comprehensively.Meanwhile,considering that not every image has a corresponding pathology report,the study leverages a text-discarding coding strategy during training to ensure that the trained model can achieve high classification accuracy despite missing text information.Experimental results show that the model performs well in CIN classification,and the classification accuracy is improved by 7-9 percentage points compared to those of traditional unimodal methods such as ResNet,ConvNeXt,and MobileViT.关键词
多模态融合/注意力机制/病理辅助诊断/深度学习/宫颈上皮内瘤变Key words
multimodal fusion/attention mechanism/pathology assisted diagnosis/deep learning/Cervical Intraepithelial Neoplasia(CIN)分类
信息技术与安全科学引用本文复制引用
FENG Saisai,GE Dongfeng,LI Tao,LIU Yijing,JI Zhihang,WANG Lin,ZHANG Mingchuan..基于多模态融合的宫颈上皮内瘤变辅助诊断[J].计算机工程,2025,51(12):304-310,7.基金项目
国家自然科学基金(62102134) （62102134）
河南省科技攻关项目(232102210048,232102211008,232102210028). （232102210048,232102211008,232102210028）
