计算机工程与应用2024，Vol.60Issue(2)：137-146,10.DOI:10.3778/j.issn.1002-8331.2207-0456
利用信息瓶颈的多模态情感分析
Multimodal Sentiment Analysis Based on Information Bottleneck
程子晨 1李彦 1葛江炜 1纠梦菲 1张敬伟1
作者信息
- 1. 天津师范大学 电子与通信工程学院,天津 300387||天津市无线移动通信与无线电能传输重点实验室,天津 300387
- 折叠
摘要
Abstract
In the field of multimodal sentiment analysis,previous research mainly focused on how to interactively fuse information from different modalities.However,based on various complex fusion strategies,the generated multimodal representation vector inevitably carries a lot of noise information irrelevant to downstream tasks,which leads to a high risk of overfitting and affects the generation of high-quality prediction results.In order to solve the above problems,accord-ing to the information bottleneck theory,this paper designs a mutual information estimation module containing two mutual information estimators,aiming to optimize the lower bound of mutual information between the multimodal representation vector and the true label,while minimizing the multimodality.The mutual information between the representation vector and the input data is used to find a concise multimodal representation vector with better predictive ability.Using MOSI and MOSEI and CH-SMIS datasets to conduct comparative experiments,the results show that the method proposed in this paper is effective.关键词
多模态情感分析/信息瓶颈理论/互信息估计Key words
multimodal sentiment analysis/information bottleneck/mutual information estimating分类
信息技术与安全科学引用本文复制引用
程子晨,李彦,葛江炜,纠梦菲,张敬伟..利用信息瓶颈的多模态情感分析[J].计算机工程与应用,2024,60(2):137-146,10.
