计算机工程与应用2024，Vol.60Issue(11)：95-104,10.DOI:10.3778/j.issn.1002-8331.2302-0064
融合改进图卷积的跨模态检索
Cross-Modal Retrieval with Improved Graph Convolution
摘要
Abstract
Aiming at the problem that existing image text cross-modal retrieval is difficult to fully exploit the local consis-tency in the mode in the common subspace,a cross-modal retrieval method based on improved graph convolution is pro-posed.In order to improve the local consistency within each mode,the modal diagram is constructed with a single sample as a node,fully mining the interactive information between features.In order to solve the problem that graph convolution network can only do shallow learning,the method of adding initial residual link and weight identity map in each layer of graph convolution is adopted to alleviate this phenomenon.In order to jointly update the central node features through higher-order and lower-order neighbor information,an improvement is proposed to reduce neighbor nodes and increase the number of layers in graph convolution network.In order to learn highly locally consistent and semantically consistent public representation,it shares the weights of common representation learning layer,and jointly optimizes the semantic constraints within the modes and the modal invariant constraints between modes in the common subspace.The experimen-tal results show that on the two cross-modal data sets of Wikipedia and Pascal sentence,the average mAP values of differ-ent retrieval tasks are 2.2%～42.1%and 3.0%～54.0%higher than the 11 existing methods.关键词
图卷积网络/跨模态检索/初始残差连接/恒等映射/邻接矩阵Key words
graph convolution network/cross-modal retrieval/initial residual connection/identity mapping/adjacency matrix分类
信息技术与安全科学引用本文复制引用
张宏图,化春键,蒋毅,俞建峰,陈莹..融合改进图卷积的跨模态检索[J].计算机工程与应用,2024,60(11):95-104,10.基金项目
国家自然科学基金(62173160). （62173160）
