计算机工程2024，Vol.50Issue(1)：289-295,7.DOI:10.19678/.issn.1000-3428.0066412
基于预训练和多模态融合的假新闻检测
Fake News Detection Based on Pre-Training and Multi-Modal Fusion
摘要
Abstract
Existing multi-modal detection models are typically characterized by a simple splicing of features from each modality and are often ineffective in modeling the correlation between modalities.Furthermore,the migration of these models to domains with sparse labels is challenging.In this paper,a PMFD model,based on pre-training and multi-modal fusion,is proposed.Initially,image raw vectors are extracted from different regions of news incidental images,which are then merged to form image guide vectors.Three distinct multimodal fusion methods are designed:early fusion,middle fusion,and post fusion.During early fusion,the text feature extractor is initialized with image bootstrap vectors,leading to the acquisition of text original vectors,which are subsequently merged into text bootstrap vectors.In the middle fusion stage,the feature representation of the modality is constructed using the modality's original vectors combined with the bootstrap vectors of other modalities.For post fusion,the feature representations of different modalities are fused to construct the feature representation of news.To enhance the model's generalization capability,PMFD is initially pre-trained on label-rich data and then fine-tuned on label-sparse data.Experimental results on public data set show that,this approach demonstrates an improvement of over 10%compared to traditional models,including CNN,LSTM,and BERT,and a 2%-3%enhancement over existing EANN,M_model multi-modal fake news detection models.关键词
假新闻检测/预训练/多模态融合/引导向量/跨模态共享特征/阶段融合Key words
fake news detection/pre-training/multi-modal fusion/bootstrap vector/cross-modal shared feature/stage fusion分类
信息技术与安全科学引用本文复制引用
周昊玮,刘勇,玄萍..基于预训练和多模态融合的假新闻检测[J].计算机工程,2024,50(1):289-295,7.基金项目
国家自然科学基金(61972135) （61972135）
黑龙江省自然科学基金(LH2020F043). （LH2020F043）
