智能系统学报2025，Vol.20Issue(2)：516-528,13.DOI:10.11992/tis.202401001
基于自适应图学习权重的多模态情感分析
Multimodal sentiment analysis based on adaptive graph learning weight
摘要
Abstract
The inconsistency in representing different modalities in multimodal sentiment analysis tasks results in signi-ficant differences in the density of emotional information between modalities.A multimodal sentiment analysis method based on adaptive graph learning weights is proposed to balance the uneven distribution of emotional information in dif-ferent modalities and reduce the redundancy of multimodal feature representations.First,different feature extraction methods are used to capture specific information within each mode.Second,different modalities are mapped to the same space through a common encoder,and cross-modal attention mechanisms are used to explicitly construct correlations between modalities.Third,the predicted values and modal representations of each modality for task classification are embedded into the adaptive graph,and the contribution of different modalities to the final classification task is learned through modal labels to dynamically adjust the weights between different modalities for adapting to changes in the dom-inant modality.Finally,an information bottleneck mechanism is introduced for denoising,aiming to learn a nonredund-ant multimodal feature representation for sentiment prediction.The proposed model is evaluated on the publicly avail-able multimodal sentiment analysis datasets.Experimental results show that its effectively improving the accuracy of multimodal sentiment analysis.关键词
多模态/情感分析/模态差异性/信息冗余/自适应图学习/跨模态注意力/相似性约束/信息瓶颈Key words
multimodal/sentiment analysis/modal differences/information redundancy/adaptive graph learning/cross modal attention/similarity constraints/information bottleneck分类
信息技术与安全科学引用本文复制引用
曲海成,徐波..基于自适应图学习权重的多模态情感分析[J].智能系统学报,2025,20(2):516-528,13.基金项目
辽宁省高等学校基本科研项目(LIKMZ20220699). （LIKMZ20220699）
