智能系统学报2024，Vol.19Issue(2)：392-400,9.DOI:10.11992/tis.202212011
融合交叉注意力的突发事件多模态中文反讽识别模型
A multimodal Chinese sarcasm detection model for emergencies based on cross attention
摘要
Abstract
Internet users often use sarcasm when discussing emergencies on social media,which complicates emotional analysis.In addition,there is a lack of research on multimodal comments,particularly those in Chinese,and their use of sarcasm on social media platforms.Therefore,it is necessary to delve deeper into sarcasm detection in multimodal Chinese content,specifically within images and text.To address this need,we propose a multimodal Chinese sarcasm detection model called the fuse cross-attention model(FCAM).This model incorporates a cross-attention mechanism to identify inconsistencies between modes.The text convolutional neural network(TextCNN)is used to extract basic fea-tures of Chinese text,while the deep residential network(ResNet)is used to extract image features.The cross-attention mechanism is used to obtain attention features from the text and image layers.The residual method is employed to estab-lish a connection between the basic text features and the text layer's attention features,as well as a link between the im-age features and the image layer's attention features.These two feature representations are fused using the attention mechanism,resulting in the sarcasm classification results through the classification layer.We have constructed a mul-timodal Chinese sarcasm data set based on Weibo comment data related to the COVID-19 pandemic in a specific region.Experimental testing on this data set confirms that FCAM holds certain advantages over the benchmark model.关键词
突发事件/社交媒体/多模态评论/中文反讽识别/中文反讽数据集/交叉注意力机制/注意力机制/情感分析Key words
emergency/social media/multimodal comment/Chinese sarcasm detection/Chinese sarcasm dataset/cross-attention mechanism/attention mechanism/sentiment analysis分类
信息技术与安全科学引用本文复制引用
胡文彬,陈龙,黄贤波,陈晨,仲兆满..融合交叉注意力的突发事件多模态中文反讽识别模型[J].智能系统学报,2024,19(2):392-400,9.基金项目
国家自然科学基金项目(72174079) （72174079）
江苏省"青蓝工程"优秀教学团队(2022-29). （2022-29）
