四川大学学报（自然科学版）2024，Vol.61Issue(2)：94-105,12.DOI:10.19907/j.0490-6756.2024.023004
基于多任务多模态学习的谣言检测框架
Rumor detection framework based on multitask multi-modal learning
摘要
Abstract
Rumor detection is the task of identifying the veracity of the information on social networks.Previ-ous studies have shown that fusing multimodal information can be helpful to rumor detection.However,these approaches have some limitations:(1)simply concatenated unimodal features without considering inter-modality relations,resulting in limited improvement in prediction performance and generalization ability.(2)did not carefully consider the composition structure of social network data,assuming it was only composed of image-text pairs and unable to handle multi-image data or missing modalities.To address these issues,we proposed a novel framework called multitask multimodal rumor detection framework(MMRDF),which con-sists of three sub-networks that generate joint multimodal representation by merging features at different lev-els and enriching intra-modal features with feature maps from different subspaces.Moreover,the joint multi-modal representation is generated by a composite convolutional fusion structure to achieve better prediction performance.MMRDF is flexible and capable of handling various types of tweets,including pure text,pure image,image-text pairs,and text with multi-images.Additionally,the MMRDF does not require extra time-delaying data such as propagation structures and response content,allowing for immediate application to ru-mor detection and reducing the time delay in debunking rumors.Experimental results on two real-world datas-ets demonstrate that our framework outperforms the state-of-the-art methods,achieving an accuracy improve-ment of 7.3%and 2.9%.Ablation experiments further validate the effectiveness of each module in the pro-posed framework.关键词
谣言检测/多模态分析/表示学习/多任务学习/神经网络Key words
Rumor detection/Multi-modal analysis/Representation learning/Multitask learning/Neural Networks分类
信息技术与安全科学引用本文复制引用
蒋方婷,梁刚..基于多任务多模态学习的谣言检测框架[J].四川大学学报（自然科学版）,2024,61(2):94-105,12.基金项目
自然科学基金联合项目(62162057) （62162057）
四川省科技厅重点研发项目(2022YFG0182) （2022YFG0182）
教育部地方项目(2020CDZG-18,2021CDLZ-12,2021CDZG-11) （2020CDZG-18,2021CDLZ-12,2021CDZG-11）
达州科技局计划项目(21ZDYF0009) （21ZDYF0009）
四川省社会科学重点研究基地——系统科学与企业发展研究中心规划项目 （）
