计算机工程2025，Vol.51Issue(11)：144-151,8.DOI:10.19678/j.issn.1000-3428.0069721
基于密集协同注意力的多模态情感分析
Multimodal Sentiment Analysis Based on Dense Co-Attention
摘要
Abstract
With the development of social networks,people are increasingly expressing their emotions through multimodal data,such as audio,text,and video.Traditional sentiment analysis methods struggle to process emotional expressions in short videos effectively,and existing multimodal sentiment analysis techniques face issues such as low accuracy and insufficient interaction between modes.To address these problems,this study proposes a Multimodal Sentiment Analysis method based on Dense Co-Attention(DCA-MSA).First,it utilizes the pre-trained Bidirectional Encoder Representations from Transformers(BERT)model,OpenFace 2.0 model,and COVAREP tool to extract features from text,video,and audio,respectively.It then employs a Bidirectional Long Short-Term Memory(BiLSTM)network to model the temporal correlations within different features separately.Finally,it integrates different features through a dense co-attention mechanism.The experimental results show that the model proposed in this paper is competitive in multimodal sentiment analysis tasks compared to some baseline models:on the CMU-MOSEI dataset,the highest increase in binary classification accuracy is 3.7 percentage points,and the highest increase in F1 value is 3.1 percentage points;on the CH-SIMS dataset,the highest increase in binary classification accuracy is 4.1 percentage points,the highest increase in three-classification accuracy is 2.8 percentage points,and the highest increase in F1 value is 3.9 percentage points.关键词
多模态/情感分析/模态交互/密集协同注意力/特征融合Key words
multimodal/sentiment analysis/modal interaction/dense co-attention/feature fusion分类
计算机与自动化引用本文复制引用
周世向,于凯..基于密集协同注意力的多模态情感分析[J].计算机工程,2025,51(11):144-151,8.基金项目
新疆维吾尔自治区社会科学基金(21BTQ162) （21BTQ162）
新疆维吾尔自治区重点研发计划项目(2023B01032). （2023B01032）
