智能系统学报2025，Vol.20Issue(6)：1461-1473,13.DOI:10.11992/tis.202503032
结合多面图像特征提取和门控融合机制的多模态方面级情感分析
Multimodal aspect-based sentiment analysis combining multifaceted image feature extraction and gated fusion mechanism
摘要
Abstract
Existing multimodal aspect-based sentiment analysis models only extract single global image features,thereby overlooking key detailed information.To address this issue,this study proposes a network model that combines multifaceted image feature extraction and a gated fusion mechanism.Specifically,a multifaceted image feature extrac-tion module is constructed in the proposed model.By leveraging cross-modal translation technology,textual descrip-tions of scenes,human faces,objects,and colors are generated from multiple sentiment-related dimensions of the image.This process achieves detailed information extraction and cross-modal information alignment.Furthermore,a gated fu-sion interaction module has been developed,incorporating a gating mechanism and interactive attention to facilitate effi-cient fusion and interaction between features.In order to address the representation gap across different modalities,se-quence information is integrated with image prompts to convert image features into the input space of the pre-trained language model(PLM).This facilitates more accurate sentiment classification.Experiments conducted on the Twitter-2015 and Twitter-2017 datasets demonstrate that compared with existing models,the proposed model achieves an aver-age improvement of 0.93%in accuracy and 0.52%in F1-score,effectively enhancing the performance of sentiment clas-sification.关键词
全局特征/多模态/方面级情感分析/文本描述/门控机制/交互注意力/图片提示/预训练语言模型Key words
global feature/multimodal/aspect-based sentiment analysis/text description/gating mechanism/cross atten-tion/image-prompt/pre-trained language model分类
信息技术与安全科学引用本文复制引用
ZHAO Xuefeng,DI Hengxi,BAI Changze,ZHONG Zhaoman,ZHONG Xiaomin..结合多面图像特征提取和门控融合机制的多模态方面级情感分析[J].智能系统学报,2025,20(6):1461-1473,13.基金项目
国家自然科学基金项目(72174079) （72174079）
江苏省"青蓝工程"优秀教学团队项目(2022-29). （2022-29）
