计算机与现代化Issue(4)：9-15,24,8.DOI:10.3969/j.issn.1006-2475.2026.04.002
结合生成对抗网络和跨模态映射融合的多模态情感分析
Multimodal Sentiment Analysis Combining Generative Adversarial Networks and Cross-modal Mapping Fusion
摘要
Abstract
In the field of smart education,multi-modal sentiment analysis technology deeply analyzes classroom audio and video data to accurately assess students'emotional states,providing a basis for optimizing teaching.However,existing methods face limitations,including insufficient feature extraction from single modality,which leads to information loss,and unstable multi-modal data fusion,resulting in suboptimal fusion representations.To address these issues,this paper proposes a multi-modal sentiment analysis method that combines adversarial networks with cross-modal mapping fusion.First,a Generative Adversarial Network(GAN)is employed to enhance the text and audio features extracted by BERT and LSTM,respectively,while DeiT is used to extract visual features,ensuring the sufficiency of features from each modality.Second,a cross-modal mapping fusion method is introduced,combining an improved cross-modal Transformer and a unique fusion gating mechanism to facilitate inter-modal information interaction and capture emotion-related information.Additionally,a Unimodal Label Generation Module(ULGM)is incorporated to help the model learn the unique characteristics of each modality.Experiments conducted on the MOSI and the MOSEI datasets show that the proposed method significantly improves sentiment analysis accuracy,thereby validating the effectiveness of the proposed model.关键词
跨模态映射融合/生成对抗网络/智慧教育/门控机制/多模态情感分析Key words
cross-modal mapping fusion/generative adversarial networks/smart education/gating mechanism/multi-modal sentiment analysis分类
信息技术与安全科学引用本文复制引用
冯广,李伟辰,黄荣灿,周垣桦,钟婷,林健忠,盘皓然..结合生成对抗网络和跨模态映射融合的多模态情感分析[J].计算机与现代化,2026,(4):9-15,24,8.基金项目
国家自然科学基金重点项目(62237001) （62237001）
广东省哲学社会科学青年项目(GD23YJY08) （GD23YJY08）
