计算机工程与应用2024，Vol.60Issue(19)：139-147,9.DOI:10.3778/j.issn.1002-8331.2311-0069
双模态双向感知下语义信息增强的多模态情感分析
Multimodal Emotional Analysis of Semantic Information Enhancement Under Bimodal Bidirec-tional Perception
摘要
Abstract
In response to the problem of uneven distribution of emotional information among different modalities,which makes it difficult to obtain deeper multimodal emotional semantic information,this paper proposes a multimodal emotion-al analysis method for semantic information enhancement under bimodal bidirectional perception.Firstly,the text visual modality and text audio modality are fused separately to capture the interdependence in bimodal discourse and obtain bidi-rectional contextual information between modalities.Secondly,considering that bimodal fusion generates more redundant information,a gating mechanism is adopted to select effective emotional features to enhance the ability to identify key emotional information.Finally,the information between multiple modalities is modeled through a cross modal informa-tion exchange mechanism to obtain semantic information enhanced modal feature vectors.The proposed model is evaluat-ed on the publicly available multimodal sentiment analysis dataset CMU-MOSI,and experimental results show that the sentiment analysis results of the model are superior to most existing advanced multimodal sentiment analysis methods,which can effectively improve the performance of sentiment analysis.关键词
多模态情感分析/双模态双向感知/门控机制/信息增强Key words
multimodal sentiment analysis/bimodal bidirectional perception/gating mechanism/information enhancement分类
信息技术与安全科学引用本文复制引用
曲海成,徐波..双模态双向感知下语义信息增强的多模态情感分析[J].计算机工程与应用,2024,60(19):139-147,9.基金项目
国家自然科学基金面上项目(42271409) （42271409）
辽宁省高等学校基本科研项目(LIKMZ20220699). （LIKMZ20220699）
