山西大学学报（自然科学版）2026，Vol.49Issue(1)：1-14,14.DOI:10.13451/j.sxu.ns.2025091
基于全局局部交互模型的多模态情感分析
Multimodal Sentiment Analysis Based on Global-local Interaction Model
摘要
Abstract
Multimodal aspect-based sentiment analysis(MABSA)is a critical research direction in the field of affective computing,aiming to integrate multimodal information,such as text,images,and audio to achieve fine-grained analysis of sentiment toward spe-cific aspects.Current research in MABSA faces challenges such as image noise interference and excessive reliance on local features,which compromise the accuracy and comprehensiveness of the analysis.To address these limitations,this paper proposes an innova-tive global-local interactive emotion analysis model(GLIEAM).On the one hand,the model employs a tandem architecture of vi-sion transformer(Vision Transformer,ViT)and generative pre-trained transformer(GPT)to generate image descriptions,which are then concatenated with original text features,significantly enhancing information fusion and enabling a more comprehensive capture of emotional cues in multimodal data.On the other hand,to mitigate image noise,a hybrid approach combining wavelet transform and non-local means is applied for image denoising.Additionally,convolutional neural networks(CNN)and tokens-to-token vision transformer(T2T-ViT)are utilized to extract local and global image features,respectively,avoiding over-reliance on local features and achieving balanced and holistic image feature extraction.Experimental results on benchmark datasets demonstrate that the pro-posed method outperforms existing approaches,the accuracy reached 78.46%on the Twitter-15 dataset and 75.21%on the Twitter-17 dataset,particularly exhibiting superior performance in low-resource scenarios.关键词
多模态方面级情感分析/视觉Transformer/特征融合/图像去噪/注意力机制Key words
multimodal aspect-based sentiment analysis/vision Transformer/feature fusion/image denoising/attention mechanism分类
信息技术与安全科学引用本文复制引用
李梦晗,仲兆满,徐俊康,陈柯含..基于全局局部交互模型的多模态情感分析[J].山西大学学报（自然科学版）,2026,49(1):1-14,14.基金项目
国家自然科学基金(72174079) （72174079）
江苏省"青蓝工程"大数据优秀教学团队(2022-29) （2022-29）
连云港市重点研发(产业前瞻与关键核心技术)项目(CG2323) （产业前瞻与关键核心技术）
