智能科学与技术学报2025，Vol.7Issue(3)：338-349,12.DOI:10.11959/j.issn.2096-6652.202521
融合注意力协同和对比学习的跨模态突发事件识别方法
A cross-modal emergency recognition method integrating attention collaboration and contrastive learning
摘要
Abstract
To address the challenges of image complexity,limited textual information,and inter-modal misleading data in cross-modal emergency recognition,a cross-modal emergency recognition method that integrates attention collaboration and contrastive learning was proposed.First,a multi-branch self-attention mechanism was integrated into an image fea-ture encoder,while a text feature encoder was augmented with external knowledge.These encoders were designed to ex-tract rich semantic features from images and contextual information from text,respectively,to overcome the limitations of single-modal representations.Next,a cross-modal attention collaboration network was introduced to capture intricate rela-tionships between image-text features,enhancing inter-modal consistency and mitigating the influence of misleading in-formation.Finally,a contrastive learning mechanism,which was optimized through a joint supervision strategy,was em-ployed to dynamically balance the contributions of image and text features.Extensive experiments conducted on the public CrisisMMD dataset and the self-constructed dataset demonstrate the superior performance of the proposed method.关键词
突发事件识别/跨模态/注意力协同/对比学习Key words
emergency recognition/cross-modal/attention collaboration/contrastive learning分类
信息技术与安全科学引用本文复制引用
黄少年,彭永涛,文沛然,刘耀..融合注意力协同和对比学习的跨模态突发事件识别方法[J].智能科学与技术学报,2025,7(3):338-349,12.基金项目
国家社会科学基金项目(No.21BTJ026) （No.21BTJ026）
湖南省社会科学成果评审委员会课题(No.XSP25YBC164)The National Social Science Foundation of China(No.21BTJ026),The Project of Hunan Social Science Achieve-ment Appraisal Committee(No.XSP25YBC164) （No.XSP25YBC164）
