智能科学与技术学报2025，Vol.7Issue(2)：257-267,11.DOI:10.11959/j.issn.2096-6652.202514
双级门控分段式多模态情绪识别方法
Dual-stage gated segmented multimodal emotion recognition method
摘要
Abstract
Multimodal emotion recognition has broad applications in mental health detection and affective computing.However,most existing methods rely on either global or local features,neglecting the joint modeling of both,which limits emotion recognition performance.To address this,a Transformer-based dual-stage gated segmented multimodal emotion recognition method(DGM).DGM adopts a segmented fusion architecture was proposed,consisting of an interaction stage and a dual-stage gating stage.In the interaction stage,the OAGL fusion strategy was employed to model global-local cross-modal interactions,improving the efficiency of feature fusion.The dual-stage gating stage integrates local and global features was designed to fully utilize emotional information.Additionally,to resolve the misalignment of local tem-poral features across modalities,a scaled dot-product-based sequence alignment method was developed to enhance fusion accuracy.Experimental were conducted on three benchmark datasets(CMU-MOSI,CMU-MOSEI,and CH-SIMS),and the results demonstrate that DGM outperforms representative algorithms on multiple datasets,validating its ability to cap-ture emotional details and its strong generalization capability.关键词
多模态情绪识别/缩放点积注意力/双级门控融合/TransformerKey words
multimodal emotion recognition/scaled dot-product attention/dual-stage gated fusion/Transformer分类
信息技术与安全科学引用本文复制引用
马飞,李树志,杨飞霞,徐光宪..双级门控分段式多模态情绪识别方法[J].智能科学与技术学报,2025,7(2):257-267,11.基金项目
辽宁省教育科学"十四五"规划课题(No.JG24DB219) （No.JG24DB219）
辽宁省科技厅自然科学基金计划面上项目(No.2023-MS-314) （No.2023-MS-314）
辽宁省教育厅高校科研业务经费项目(No.LJ242410147006) （No.LJ242410147006）
辽宁工程技术大学GPU资源支持项目 The Education Science"14th Five-Year Plan"of Liaoning Province(No.JG24DB219),The Natural Science Foundation of Science and Technology of Liaoning Provincial Department(No.2023-MS-314),The Scientific Research Project of Colleges from Liaoning Department of Education(P.R.C)(No.LJ242410147006),GPU Resource from LNTU （No.JG24DB219）
