计算机应用研究2026，Vol.43Issue(5)：1315-1321,7.DOI:10.19734/j.issn.1001-3695.2025.10.0396
基于分层门控与双重对比监督机制的多模态命名实体识别方法
Multimodal named entity recognition approach based on hierarchical gating and dual contrastive supervision mechanisms
摘要
Abstract
The multimodal named entity recognition task still faces shortcomings in deep cross-modal interaction and entity-level alignment supervision,making it difficult for models to fully leverage fine-grained semantic correlations between images and text.To address this issue,this paper proposed a multimodal NER model,HCMCL,which integrated a hierarchical gating mechanism with dual contrastive supervision.The model firstly introduced a bidirectional state-space structure to enhance the ability to model long-range dependencies in text sequences and visual region features.Next,it designed a multi-level cross-modal gating structure to adaptively regulate the way visual information was injected at different semantic levels,achieving fine-grained cross-modal feature fusion.To further address the weak alignment between images and text,it constructed a dual contrastive supervision strategy at both the sentence and entity levels to explicitly constrain modality alignment.Experimental results show that HCMCL achieves F1 scores of 76.88％and 87.96％on the Twitter-2015 and Twitter-2017 datasets,respec-tively,significantly outperforming current mainstream multimodal methods.Ablation studies also verify the effectiveness of each module.The study demonstrates that the proposed method effectively enhances the learning of cross-modal semantic asso-ciations and improves entity recognition performance in complex image-text scenarios.关键词
多模态命名实体识别/分层门控/跨模态对齐/对比监督Key words
multi-modal named entity recognition/hierarchical gating/cross-modal alignment/contrastive supervision分类
信息技术与安全科学引用本文复制引用
刘思婷,徐艳丽..基于分层门控与双重对比监督机制的多模态命名实体识别方法[J].计算机应用研究,2026,43(5):1315-1321,7.基金项目
国家自然科学基金资助项目(62271303) （62271303）
中国上海市教育委员会创新计划资助项目(2021-01-07-00-10-E00121) （2021-01-07-00-10-E00121）
上海自然科学基金会资助项目(20ZR1423200) （20ZR1423200）
