重庆理工大学学报2024，Vol.38Issue(1)：169-179,11.DOI:10.3969/j.issn.1674-8425(z).2024.01.019
结合图卷积网络的多模态仇恨迷因识别研究
Research on multimodal hate meme recognition based on graph convolutional network
摘要
Abstract
Memes exist in the form of images and texts and are used to describe hate speeches,rumors that spread among users on the Internet.They often use web entities such as popular figures,events,or historical figures to express hate emotions.These implicit emotional expressions are worth academic attention,but web entities are mostly ignored by existing meme identification methods.To address the problem,this paper proposes a meme recognition method based on graph convolutional network.Specifically,the web entity information contained in the image is first extracted.The web entity modality and the text modality are fused by a graph convolutional network.An external dictionary is employed to measure the relationship between the web entity and the meme text from multiple perspectives when building cross-domain graph.Then,the text and image modalities are interacted through the attention module.Finally,the self-distillation technology is employed to improve the model's information utilization rate.Our experimental results on the Hateful Memes dataset and the MAMI dataset reach an accuracy of 76.03％and 73.9％respectively,and the performance is superior to the existing SOTA model.关键词
迷因识别/网络实体识别/隐式情感分析/图卷积网络Key words
meme recognition/web entity detection/implicit sentiment analysis/graph convolutional network分类
信息技术与安全科学引用本文复制引用
刘旭东,杨亮,张冬瑜,林鸿飞..结合图卷积网络的多模态仇恨迷因识别研究[J].重庆理工大学学报,2024,38(1):169-179,11.基金项目
国家自然科学基金项目(62076046,62076051) （62076046,62076051）
