计算机应用研究2024，Vol.41Issue(4)：1239-1246,8.DOI:10.19734/j.issn.1001-3695.2023.08.0370
基于二重语义相关性图卷积网络的跨模态检索方法
Dual semantic correlation graph convolutional networks for cross-modal retrieval
摘要
Abstract
With the continuous development of deep neural networks,significant progress has been made in the construction of cross-modal retrieval models.Cross-modal retrieval methods based on GCN have shown promising results in capturing semantic correlations in data,thus attracting increasing attention.However,most recent research focuses on incorporating correlations between labels and between samples into cross-modal representations,while the impact of correlations between label sets is neglected.In multi-label scenarios,the correlations between label sets can effectively describe semantic relationships between corresponding samples.Therefore,exploring the multi-label correlations and integrating it into cross-modal representations is important for improving the performance of cross-modal retrieval models.This paper proposed a dual semantic correlation graph convolutional networks(DSCGCN)cross-modal retrieval method.This method utilized GCN to explore the semantic correla-tions between labels and between multi-labels adaptively,and integrated the learned dual semantic correlations into the com-mon representations.Additionally,it designed a multi-label similarity loss to make the similarities between the common repre-sentations more close to the semantic similarities.Experimental results on the NUS-WIDE,MIRFlickr-25 K,and MS-COCO datasets demonstrate that because of multi-label correlations,DSCGCN achieves satisfactory retrieval performance.关键词
语义相关性/自适应相关性矩阵/图卷积网络/跨模态检索Key words
semantic correlation/adaptive correlation matrix/graph convolutional network(GCN)/cross-modal retrieval分类
信息技术与安全科学引用本文复制引用
刘佳楠,范晶晶,赵建光,朱杰..基于二重语义相关性图卷积网络的跨模态检索方法[J].计算机应用研究,2024,41(4):1239-1246,8.基金项目
河北省自然科学基金资助项目(F2022511001) （F2022511001）
河北省高等学校科学技术研究项目(ZC2022070) （ZC2022070）
河北大学高层次人才科研启动项目(521100223212) （521100223212）
张家口市市级科技计划财政资助项目(2311010A) （2311010A）
张家口市2022年度基础研究专项资助项目(2221008A) （2221008A）
河北建筑工程学院2024年校级研究生创新基金资助项目(XY2024068) （XY2024068）
