华东师范大学学报（自然科学版）Issue(2)：131-142,12.DOI:10.3969/j.issn.1000-5641.2024.02.014
基于解耦常识性关联的图像描述生成算法
An image caption generation algorithm based on decoupling commonsense association
刘家伟 1林欣1
作者信息
- 1. 华东师范大学计算机科学与技术学院,上海 200062
- 折叠
摘要
Abstract
The image caption generation algorithm based on decoupling commonsense association aims to eliminate the interference of commonsense association between various types of entities on the model reasoning,and improve the fluency and accuracy of the generated description.Aiming at the relationship sentences in the current image description that conform to common sense but do not conform to the image content,the algorithm first uses a novel training method to improve the attention of the relationship detection model to the real relationship in the image and improve the accuracy of relationship reasoning.Then,a relation-aware entity interaction method was used to carry out targeted information interaction for entities with relationships,and the relationship information was strengthened.The experimental results show that the proposed algorithm can correct some commonsense false relationships,generate more accurate image captions,and obtain better experimental results on various evaluation indicators.关键词
图像描述生成/解耦常识性关联/注意力机制Key words
image captioning/decoupling commonsense association/attention分类
信息技术与安全科学引用本文复制引用
刘家伟,林欣..基于解耦常识性关联的图像描述生成算法[J].华东师范大学学报（自然科学版）,2024,(2):131-142,12.
