广西师范大学学报（自然科学版）2025，Vol.43Issue(3)：57-71,15.DOI:10.16088/j.issn.1001-6600.2024071702
差异特征导向的解耦多模态情感分析
A Dissimilarity Feature-Driven Decoupled Multimodal Sentiment Analysis
摘要
Abstract
Feature decomposition method decomposes features from different modalities into similarity and dissimilarity features.Due to the decoupled dissimilarity features containing both the diversity and the unique information,they show evident distribution discrepancies.Previous feature decomposition methods have overlooked the inherent contradictions in dissimilarity features,resulting in a decrease in prediction accuracy.To address this issue,a dissimilarity feature-driven decomposition network(DFDDN)for multimodal sentiment analysis is proposed.Firstly,feature extract module is used to extract and amplify features,which not only eliminate visual and audio noise but also facilitate the capture of complementary information between modalities.Secondly,different encoders are used to decouple the features,and a multimodal transformer is used to mitigate the differences in dissimilarity features.Finally,loss functions are used for optimization.Extensive experiments on two widely-used multimodal sentiment analysis datasets demonstrate the accuracy and robustness of this model,transcending SOTA performance.关键词
多模态情感分析/特征解耦/预训练BERT/对比学习/表示学习Key words
multimodal sentiment analysis/feature decomposition/pretraining BERT/representation learning/contrastive learning分类
信息技术与安全科学引用本文复制引用
李志欣,刘鸣琦..差异特征导向的解耦多模态情感分析[J].广西师范大学学报（自然科学版）,2025,43(3):57-71,15.基金项目
国家自然科学基金(62276073,61966004) （62276073,61966004）
广西自然科学基金(2019GXNSFDA245018) （2019GXNSFDA245018）
广西研究生教育创新计划(YCBZ2024115) （YCBZ2024115）
广西"八桂学者"工程专项基金 （）
