计算机应用研究2026，Vol.43Issue(9)：2605-2612,8.DOI:10.19734/j.issn.1001-3695.2026.02.0021
一种基于证据驱动的多模态对话混合情绪状态转移模型
Evidence-driven hybrid emotion state transition model for multimodal emotion recognition in conversation
摘要
Abstract
Multimodal conversational emotion recognition aims to predict the emotional state of each utterance in a dialogue by fusing information from multiple modalities,such as text,audio,and video.In this task,emotional states typically exhibit dy-namic evolution characteristics,while information inconsistencies across modalities introduce significant interference,which jointly increases the complexity of emotion recognition.To address the above issues,this study proposed an evidence-driven hybrid emotion state transition model.This model fused textual,acoustic and visual modalities to capture feature variations be-tween adjacent utterances,and fused intra-utterance local features with inter-utterance dynamic changes to construct evidence representations that reflected the intensity of emotional variation.Based on the obtained evidence representations,and com-bined with modality conflicts and speaker differences,the evidence representations guided a hybrid emotion state transition mechanism to adaptively regulate the two evolution patterns of continuity and abruptness.The model further combined the de-rived emotional states with dialogue context to complete utterance-level emotion prediction.Experimental results show that the proposed model achieves accuracies of 74.12％and 67.32％,and F1 scores of 74.03％and 66.11％on IEMOCAP and MELD,the benchmark datasets for multimodal conversational emotion recognition,which represents a significant improvement over existing baseline models.关键词
多模态对话情感识别/多模态融合/情绪动态建模/证据驱动状态转移Key words
multimodal conversational emotion recognition/multimodal fusion/emotion dynamics modeling/evidence-driven state transition分类
信息技术与安全科学引用本文复制引用
徐超,张雨,孙凯丽,浦悦..一种基于证据驱动的多模态对话混合情绪状态转移模型[J].计算机应用研究,2026,43(9):2605-2612,8.基金项目
国家自然科学基金面上项目(62472227) （62472227）
国家自然科学青年基金资助项目(62501289) （62501289）
江苏省高校自然科学研究重大项目(24KJA520005) （24KJA520005）
江苏省高等学校自然科学研究面上项目(25KJB520020) （25KJB520020）
