深圳大学学报（理工版）2024，Vol.41Issue(3)：283-292,10.DOI:10.3724/SP.J.1249.2024.03283
动态调整语义的词性加权多模态情感分析模型
Part of speech weighted multi-modal emotion analysis model with dynamic adjustment of semantic representation
摘要
Abstract
In order to better utilize of the semantic information contained in the part of speech of words and the contextual information of unnatural language accompanying the appearance of words,a part of speech weighted multi-modal sentiment analysis model with dynamic adjustment of semantic representation(PM-DS)is proposed.The PM-DS model takes natural language as the main body,and uses bidirectional encoder representation from transformer model,generalized autoregressive pre-training model for language understanding(XLNet)and a robustly optimized BERT pretraining approach(RoBERTa)to embed words into text patterns,respectively.A dynamic semantic adjustment module is created to effectively combine natural language and unnatural language information.The part of speech weighting module is designed to extract the part of speech of words and assigned weights to optimize sentiment discrimination.Comparative experimental results with the current advanced models such as tensor fusion network and low-rank multimodal fusion show that the average absolute errors of PW-DS model on public data sets CMU-MOSI and CMU-MOSEI are 0.607 and 0.510,respectively,and the binary classification accuracies are 89.02%and 86.93%,respectively,which is better than the models in the comparative experiments.The effects of different modules on the model are also analyzed through ablation experiments.The experimental results demonstrate that the proposed model is effective to deal with the problem of multi-modal emotion analysis.关键词
人工智能/多模态情感分析/动态调整语义/词性加权/多模态向量位置可视化/词性加权可视化Key words
artificial intelligence/multimodal sentiment analysis/dynamical adjustment semantic/part of speech weighting/visualization of multimodal vector position/visualization of part of speech weights分类
信息技术与安全科学引用本文复制引用
花强,陈卓,张峰,董春茹..动态调整语义的词性加权多模态情感分析模型[J].深圳大学学报（理工版）,2024,41(3):283-292,10.基金项目
The Innovation Capacity Enhancement Program-Science and Technology Platform Project of Hebei Province(22567623H) 河北省创新能力提升计划科技平台资助项目(22567623H) （22567623H）
