现代电子技术2025，Vol.48Issue(13)：145-151,7.DOI:10.16652/j.issn.1004-373x.2025.13.021
基于图像知识增强的中文多模态反讽检测方法
Chinese multimodal irony detection method based on image knowledge enhancement
摘要
Abstract
With the rapid development of social media and online content,the use of irony has become common in the online communication and information dissemination.However,the traditional text analysis methods often fail to capture the meaning of irony accurately,and relying solely on textual information has limitations and is of instability.In this paper,a Chinese multimodal irony dataset is constructed.The dataset includes 5 964 annotated data samples,including two modes of text and image.The images play an important role in multimodal irony detection tasks.In order to fully explore the hidden information in images,the image captioning generation model ViT-GPT2-image-captioning is used to generate the description information of the image for image knowledge enhancement,so as to enhance the understanding and cognition of the image.Moreover,a multimodal attention network model CMANet that integrates modal information for irony detection is proposed to get rid of the insufficient information correlation between modes and lack of data in the process of multi-modal data fusion.Experimental verification was performed on the dataset.The results show that the F1-score of the proposed CMANet model has been improved by 1.49%and its accuracy by 1.89%in comparison with those of the baseline model.关键词
多模态/反讽检测/注意力机制/跨模态/深度学习/融合网络Key words
multimodal/irony detection/attention mechanism/cross-modal/deep learning/network fusion分类
电子信息工程引用本文复制引用
李悦莹,曹晖,张积赛,夏啸天..基于图像知识增强的中文多模态反讽检测方法[J].现代电子技术,2025,48(13):145-151,7.基金项目
甘肃省中央引导地方科技发展资金项目(25ZYJA034) （25ZYJA034）
甘肃省教育教学成果培育项目(2023GSJXCGPY-60) （2023GSJXCGPY-60）
西北民族大学研究生教育教学改革项目(2024JGYB067) （2024JGYB067）
西北民族大学计算机应用技术创新团队项目 （）
