电子学报2025，Vol.53Issue(11)：3983-3995,13.DOI:10.12263/DZXB.20250554
混合量子与图神经网络的多模态情感分析方法
A Hybrid Quantum-Graph Neural Network for Multimodal Sentiment Analysis
摘要
Abstract
Multimodal sentiment analysis(MSA)is one of the most promising technologies in the field of affective computing.Visual,acoustic,and textual modalities encode most human emotional features.Integrating them yields a finer,multidimensional representation of subjective affect.However,achieving accurate and robust sentiment analysis still faces significant challenges.When the sentiment feature subsets extracted from each modality differ in element quantity or tempo-ral alignment,an effective strategy for selecting representative emotional features is essential to prevent key features from being overlooked or over-extracted,thereby ensuring the reliability of subsequent fusion analysis.Direct fusion of represen-tative features across modalities often fails to fully exploit information transmission and complementarity,which can cause excessive reliance on a single modality's semantic representation and lead to overfitting or misclassification.Furthermore,human emotional expression exhibits modality heterogeneity and inconsistency,often resulting in uneven feature distribu-tions and polarity ambiguity.Algorithmic models must not only capture cross-modal complementary information and fine-grained correlations but also suppress redundant features that interfere with emotional discrimination.The presence of a"se-mantic gap"in data fusion further limits result stability.To address these issues,this paper proposes a hybrid quantum-graph neural network,inspired by multi-scale temporal representation and qubit-based polymorphic encoding.First,a topo-logical graph network of representative sequences is constructed to capture dynamic relationships among feature nodes,and a multi-head graph attention mechanism is introduced to adaptively adjust node and edge weights,ensuring reliable selec-tion of critical sentiment features.Then,a quantum sentiment feature computation network is designed,mapping multimod-al features into a high-dimensional Hilbert space via quantum encoding.Leveraging quantum superposition and entangle-ment,the model enhances deep intermodal coupling and dependency modeling.Through quantum measurement,superposed states collapse into specific eigenstates,establishing a correspondence between quantum states and sentiment features,and yielding more discriminative multimodal fusion representations.Finally,single-modal and multimodal predictions are for-mulated as multiple subtasks under a multitask collaborative optimization framework.Pseudo-label generation and shared representations improve task-specific performance,while a joint multitask loss mitigates inconsistencies among modality representations,enhancing the model's generalization ability.Experimental results on the CMU-MOSI,CH-SIMS,and CMU-MOSEI benchmark datasets demonstrate that,compared with conventional baselines,the proposed method improves binary classification accuracy by 1.5%～8.7%,five-class accuracy by 3.3%～10.7%,and seven-class accuracy by 1.5%～14.5%.The F1 score increases by up to 8.5 points,the pearson correlation coefficient improves by up to 0.146,and the mean absolute error decreases by up to 0.304.关键词
多模态情感分析/图神经网络/量子机器学习/跨模态信息融合/多任务优化Key words
multimodal sentiment analysis/graph neural network/quantum machine learning/cross-modal informa-tion fusion/multitask optimization分类
信息技术与安全科学引用本文复制引用
李兴广,蔡禹健,崔炜,李劲松,张莹瑀..混合量子与图神经网络的多模态情感分析方法[J].电子学报,2025,53(11):3983-3995,13.基金项目
吉林省科技厅项目(No.20250102225JC) Jilin Provincial Department of Science and Technology Project(No.20250102225JC) （No.20250102225JC）
