华南理工大学学报（自然科学版）2025，Vol.53Issue(11)：18-26,9.DOI:10.12141/j.issn.1000-565X.240598
基于邻居信息聚合的无配对跨模态检索重排序
Unpaired Cross-Modal Retrieval Re-Ranking Based on Neighbor Information Aggregation
摘要
Abstract
As a post-processing technique,re-ranking has demonstrated significant effectiveness in cross-modal re-trieval tasks.By mining and processing the information between initial ranking lists,re-ranking process effectively improves retrieval accuracy.The current mainstream cross-modal retrieval re-ranking methods re-rank the initial list based on paired datasets.However,they have poor flexibility because they cannot be easily plugged into exist-ing systems without modifying the original framework and retraining,which makes it difficult to transfer them to other frameworks.Moreover,they cannot be applied in unpaired scenarios.At present,cross-modal retrieval has achieved significant progress by relying on large-scale paired datasets,but it overlooks the problem that labeling such large-scale datasets in practical scenarios requires substantial resources.To address these issues,this paper proposes an unpaired cross-modal retrieval re-ranking method based on neighbor information aggregation.The method improves retrieval performance by mining and utilizing the neighbor information of samples,pushing incor-rect answers away from the query input.It searches for local neighbors in the Euclidean neighborhood and for global neighbor expressions through collaborative expression,and then integrates these two types of neighbor infor-mation to generate new features for re-calculating semantic similarity with the retrieval input,thus completing a re-ranking process.Finally,the proposed method is applied as a post-processing technique in several cross-modal re-trieval model frameworks and is tested on MSCOCO dataset,with its effectiveness and superiority over other re-ranking methods being demonstrated.关键词
跨模态检索/重排序方法/邻居信息聚合/全局语义邻居/局部语义邻居Key words
cross-modal retrieval/re-ranking method/neighbor information aggregation/global semantic neighbor/local semantic neighbor分类
信息技术与安全科学引用本文复制引用
沃焱,梁展扬..基于邻居信息聚合的无配对跨模态检索重排序[J].华南理工大学学报（自然科学版）,2025,53(11):18-26,9.基金项目
广东省自然科学基金项目(2025A1515011905) Supported by the Natural Science Foundation of Guangdong Province(2025A1515011905) （2025A1515011905）
