广西师范大学学报（自然科学版）2024，Vol.42Issue(2)：84-93,10.DOI:10.16088/j.issn.1001-6600.2023052701
基于跨模态交叉注意力网络的多模态情感分析方法
Multimodal Sentiment Analysis Based on Cross-Modal Cross-Attention Network
摘要
Abstract
Exploiting intra-modal and inter-modal information is helpful for improving the performance of multimodal sen-timent analysis.So,a multimodal sentiment analysis based on cross-modal cross-attention network is proposed.Firstly,VGG-16 network is utilized to map the multimodal data into the global feature space.Simultaneously,the Swin Transformer network is used to map the multimodal data into the local feature space.And the intra-modal self-attention and inter-modal cross-attention features are constructed.Then,a cross-modal cross-attention fusion module is designed to achieve the deep fusion of the intra-modal and inter-modal features,enhancing the represen-tation reliability of the multimodal feature.Finally,the softmax function is used to obtain the results of the sentiment analysis.The experimental results on two open source datasets CMU-MOSI and CMU-MSOEI show that the proposed model can achieve an accuracy of 45.9％and 54.1％respectively in the seven-classification task.Compared with the current classical MCGMF model,the accuracy of the proposed model has improved by 0.66％and 2.46％,and the overall performance improvement is significant.关键词
情感分析/多模态/跨模态交叉注意力/自注意力/局部和全局特征Key words
sentiment analysis/multimodal/cross-modal cross-attention/self-attention/global and local feature分类
计算机与自动化引用本文复制引用
王旭阳,王常瑞,张金峰,邢梦怡..基于跨模态交叉注意力网络的多模态情感分析方法[J].广西师范大学学报（自然科学版）,2024,42(2):84-93,10.基金项目
国家自然科学基金(62161019) （62161019）
