计算机应用研究2025，Vol.42Issue(7)：1972-1977,6.DOI:10.19734/j.issn.1001-3695.2024.12.0497
基于图文对比融合的图像人物情感识别
Human emotion recognition in images based on text-image contrastive fusion
摘要
Abstract
Context-based recognition of human emotions in images has become an increasingly popular task in recent years,with application value in many fields.Most existing methods only encode the human subject and the background separately,ex-tracting isolated features for simple interaction,lacking an effective feature fusion mechanism between the subject and the con-textual background.Aimed to address the issue of the interaction between complex backgrounds and the human subject,this pa-per proposed a new network for human emotion recognition in images based on text-image contrastive fusion.Firstly,it designed prompt words to extract textual descriptions of the emotional state between the contextual background and the target human sub-ject by fully utilized the extensive social context information and reasoning capabilities of large visual-language models.Second-ly,it proposed a text-image contrastive fusion module,which fused the cropped target human subject image features with the text description features obtained based on the prompt words through this module.Finally,the fusion algorithm introduced a contrastive loss function to unify the representation of image encoding and text encoding,allowing for more accurate capture of effective emotional expressions during fusion.Experimental results show that the network can learn more effective emotional fea-ture representations,and the network achieves superior results on the EMOTIC dataset with an mAP of 37.30％.The proposed method better integrates the features of the human subject and the background in the image,thereby improving the accuracy of human emotion recognition in images.关键词
情感识别/视觉语言模型/情境感知/多模态融合Key words
emotion recognition/vision-language model/context awareness/multimodal fusion分类
信息技术与安全科学引用本文复制引用
田雨乐,王一丁..基于图文对比融合的图像人物情感识别[J].计算机应用研究,2025,42(7):1972-1977,6.基金项目
国家自然科学基金资助项目(62276018) （62276018）
