湖南大学学报（自然科学版）2025，Vol.52Issue(4)：68-78,11.DOI:10.16339/j.cnki.hdxbzkb.2025267
基于孪生网络的特征融合位移RGB-T目标跟踪
FSSiamNet:Feature Fusion Shift Siamese Network for RGB-T Target Tracking
摘要
Abstract
To solve the problems of the existing target tracking algorithms,such as inability to extract deep-level features,failure to fully exploit cross-modal information,and weak representation of target features,a feature fusion shift Siamese network for RGB-T target tracking is proposed.First,a target tracking framework based on the visible modal SiameseRPN++is designed to extend the infrared modal branch,in order to obtain a multimodal target tracking framework.Moreover,the improved ResNet50 network with adjusted stride as a feature extraction network enables the acquisition of deep-level features of the target.Subsequently,a multimodal feature interactive learning module(FIM)is designed to leverage the discriminative information from one modality to guide the learning process of target appearance features in the other modality.By mining the cross-modal information within the feature space and channels,the module enhances the network's attention towards foreground information.Thereafter,a multimode feature fusion module(FAM)is designed,which calculates the degree of feature fusion between the input visible light image and the infrared image,enabling spatial fusion of significant features from different modalities to effectively eliminate redundant information and reconstructing multimodal images by employing a cascade fusion strategy.Finally,a feature space shift module(FSM)is designed,which divides the feature maps of the infrared modal branches and shifts them in four different directions to enhance the edge representation of the heat source target.Extensive experiments on two RGB-T datasets thoroughly validate the effectiveness of the proposed algorithm,while ablation experiments demonstrate the superiority of each designed module.关键词
RGB-T跟踪/多模特征融合模块/特征空间位移模块/特征交互学习模块Key words
RGB-T tracking/multi mode feature fusion module/feature space shift module/feature interactive learning module分类
信息技术与安全科学引用本文复制引用
李海燕,曹永辉,郎恂,李海江..基于孪生网络的特征融合位移RGB-T目标跟踪[J].湖南大学学报（自然科学版）,2025,52(4):68-78,11.基金项目
国家自然科学基金资助项目(62166048),National Natural Science Foundation of China(62166048) （62166048）
云南省万人计划"云岭教学名师","Yunling Famous Teacher"of Yunnan 10 000 Persons Program （）
云南省高校重点实验室建设计划资助项目(202101AS070031),Key Laboratory Construction Plan of Universities in Yunnan Province(202101AS070031) （202101AS070031）
云南大学第十四届研究生科研创新项目(KC-22221737),The 14th Graduate Student Research and Innovation Project at Yunnan University(KC-22221737) （KC-22221737）
