南方医科大学学报2024，Vol.44Issue(3)：585-593,9.DOI:10.12122/j.issn.1673-4254.2024.03.21
基于多模态多示例学习的免疫介导性肾小球疾病自动分类方法
Automatic classification of immune-mediated glomerular diseases based on multi-modal multi-instance learning
摘要
Abstract
Objective To develop a multi-modal deep learning method for automatic classification of immune-mediated glomerular diseases based on images of optical microscopy(OM),immunofluorescence microscopy(IM),and transmission electron microscopy(TEM).Methods We retrospectively collected the pathological images from 273 patients and constructed a multi-modal multi-instance model for classification of 3 immune-mediated glomerular diseases,namely immunoglobulin A nephropathy(IgAN),membranous nephropathy(MN),and lupus nephritis(LN).This model adopts an instance-level multi-instance learning(I-MIL)method to select the TEM images for multi-modal feature fusion with the OM images and IM images of the same patient.By comparing this model with unimodal and bimodal models,we explored different combinations of the 3 modalities and the optimal methods for modal feature fusion.Results The multi-modal multi-instance model combining OM,IM,and TEM images had a disease classification accuracy of(88.34±2.12)%,superior to that of the optimal unimodal model[(87.08±4.25)%]and that of the optimal bimodal model[(87.92±3.06)%].Conclusion This multi-modal multi-instance model based on OM,IM,and TEM images can achieve automatic classification of immune-mediated glomerular diseases with a good classification accuracy.关键词
肾活检病理/肾小球疾病/深度学习/多模态融合/多示例学习Key words
renal biopsy pathology/glomerular disease/deep learning/multi-modal fusion/multi-instance learning引用本文复制引用
龙楷兴,翁丹仪,耿舰,路艳蒙,周志涛,曹蕾..基于多模态多示例学习的免疫介导性肾小球疾病自动分类方法[J].南方医科大学学报,2024,44(3):585-593,9.基金项目
国家自然科学基金(32071368)Supported by National Natural Science Foundation of China(32071368). （32071368）
