计算机应用研究2025，Vol.42Issue(7)：1978-1985,8.DOI:10.19734/j.issn.1001-3695.2024.11.0487
基于深度特征交互与层次化多模态融合的情感识别模型
Deep feature interaction and hierarchical multimodal fusion for emotion recognition
摘要
Abstract
Multimodal emotion recognition has recently become an important research direction in affective computing,aiming to more accurately recognize and understand human emotional states by integrating various modalities such as speech and text.However,existing methods lack the processing of inter-modal correlations during feature extraction and overlook multi-scale emotional cues during feature fusion.To address these issues,this study proposed a deep feature interaction and hierarchical multimodal fusion emotion recognition model(DFIHMF).In the feature extraction stage,the model enhanced interactions be-tween different modalities and extracted multi-scale information by introducing local knowledge tokens(LKT)and cross-modal interaction tokens(CIT).In the feature fusion stage,the model integrated complex multimodal features and multi-scale emo-tional cues using a hierarchical fusion strategy.Experimental results on the MOSI and MOSEI datasets show that the model achieves accuracy rates of 45.6％and 53.5％on the ACC7 evaluation metric,demonstrating that the proposed method outper-forms existing technologies in multimodal emotion recognition tasks.关键词
多模态情感识别/层次化融合/多尺度融合/特征融合Key words
multimodal emotion recognition/hierarchical fusion/multi-scale fusion/feature fusion分类
信息技术与安全科学引用本文复制引用
王健,赵小明,王成龙,张石清,赵舒畅..基于深度特征交互与层次化多模态融合的情感识别模型[J].计算机应用研究,2025,42(7):1978-1985,8.基金项目
国家自然科学基金资助项目(62276180) （62276180）
