计算机工程与应用2026，Vol.62Issue(5)：272-280,9.DOI:10.3778/j.issn.1002-8331.2410-0363
基于文本层次化信息增强的多模态情感分析
Multimodal Sentiment Analysis Based on Text Hierarchical Information Enhancement
摘要
Abstract
Multimodal sentiment analysis aims to recognize human emotions from textual,visual and audio information.Previous approaches have not fully considered the use of textual features to enhance the emotional information present in the visual and audio modalities,as well as the correlation between the different modalities.To address these issues,this paper proposes a multimodal sentiment analysis model based on text hierarchical information enhancement(THIE).The model first uses Transformer to extract textual,visual and audio modal features in order to get important semantic information.Then,it designs a text hierarchical information enhancement module,which utilizes textual modalities at different levels to fuse relevant emotional information in visual and audio modalities,respectively,so as to achieve the effect of emotional enhancement.Finally,it proposes a multimodal fusion module based on cross-modal attention and gating mechanism,which is used to enhance the correlation existing between different feature information,increase the weight of important features,and suppress relatively unimportant features,thus reducing redundant information.Experiments on the public datasets CMU-MOSI,CMU-MOSEI and CH-SIMS show that the model has improved performance compared with main-stream methods.关键词
层次化信息增强/跨模态注意力机制/门控机制Key words
hierarchical information enhancement/cross-modal attention mechanisms/gating mechanism分类
信息技术与安全科学引用本文复制引用
程艳,房成兴,文煜..基于文本层次化信息增强的多模态情感分析[J].计算机工程与应用,2026,62(5):272-280,9.基金项目
国家自然科学基金(62167006) （62167006）
江西省科技创新基地——智能信息处理与情感计算江西省重点实验室项目(20242BCC32021) （20242BCC32021）
江西省主要学科学术和技术带头人培养计划——领军人才项目(20213BCJL22047) （20213BCJL22047）
国家社会科学基金重点项目(20AXW009) （20AXW009）
江西省自然科学基金(20212BAB202017). （20212BAB202017）
