计算机应用研究2023，Vol.40Issue(12)：3816-3820,5.DOI:10.19734/j.issn.1001-3695.2023.04.0217
基于感知重采样和多模态融合的连续情感识别
Continuous emotion recognition based on perceiver resampling and multimodal fusion
摘要
Abstract
Emotion recognition plays a crucial role in human-computer interaction,and continuous emotion recognition has gained significant attention due to its ability to capture a broader range of emotions,including more subtle ones.In the field of multimodal continuous emotion recognition,this paper proposed a continuous emotion recognition method based on perceiver resampling and multimodal fusion for the problems that the temporal series information obtained by the existing methods con-tains more redundancy and the obtained multimodal interactive information is not comprehensive.Firstly,the perceiver resam-pling module removed redundant information,focused on key information,compressed the key features with temporal relation-ships into hidden vectors,and reduced the computational complexity of the later fusion.Secondly,the multimodal fusion module captured the interactive information between modalities through cross-attention mechanism,and used the self-attention mecha-nism to obtain the hidden information within each modality,so as to make the feature information richer and more comprehen-sive.The mean CCC values of arousal and valence on the Ulm-TSST and Aff-Wild2 datasets are 63.62％and 50.09％,re-spectively,which prove the effectiveness of the model.关键词
情感识别/感知重采样/多模态融合/注意力机制Key words
emotion recognition/perceiver resampling/multimodal fusion/attention mechanism分类
信息技术与安全科学引用本文复制引用
李健,张倩,陈海丰,李晶,王丽燕,裴二成..基于感知重采样和多模态融合的连续情感识别[J].计算机应用研究,2023,40(12):3816-3820,5.基金项目
陕西科技大学博士科研启动基金资助项目(126022325) （126022325）
陕西省自然科学基础研究计划资助项目(grant 2022JQ-662) （grant 2022JQ-662）
