基于多任务多模态学习的谣言检测框架OA北大核心CSTPCD

Rumor detection is the task of identifying the veracity of the information on social networks.Previ-ous studies have shown that fusing multimodal information can be helpful to rumor detection.However,these approaches have some limitations:(1)simply concatenated unimodal features without considering inter-modality relations,resulting in limited improvement in prediction performance and generalization ability.(2)did not carefully consider the composition structure of social network data,assuming it was only composed of image-text pairs and unable to handle multi-image data or missing modalities.To address these issues,we proposed a novel framework called multitask multimodal rumor detection framework(MMRDF),which con-sists of three sub-networks that generate joint multimodal representation by merging features at different lev-els and enriching intra-modal features with feature maps from different subspaces.Moreover,the joint multi-modal representation is generated by a composite convolutional fusion structure to achieve better prediction performance.MMRDF is flexible and capable of handling various types of tweets,including pure text,pure image,image-text pairs,and text with multi-images.Additionally,the MMRDF does not require extra time-delaying data such as propagation structures and response content,allowing for immediate application to ru-mor detection and reducing the time delay in debunking rumors.Experimental results on two real-world datas-ets demonstrate that our framework outperforms the state-of-the-art methods,achieving an accuracy improve-ment of 7.3%and 2.9%.Ablation experiments further validate the effectiveness of each module in the pro-posed framework.