光学精密工程2025，Vol.33Issue(12)：1940-1954,15.DOI:10.37188/OPE.20253312.1940
多模态特征融合的RGB-T目标跟踪网络
RGB-T tracking network based on multi-modal feature fusion
摘要
Abstract
In recent years,RGB-T tracking methods have been widely used in visual tracking tasks due to the complementarity of visible image and thermal infrared images.However,the existing RGB-T moving target tracking methods have not yet made full use of the complementary information between the two mo-dalities,which limits the performance of the tracker.The existing Transformer-based RGB-T tracking al-gorithms are still short of direct interaction between the two modalities,which limits the full use of the original semantic information of RGB and TIR modalities.To solve this problem,the paper proposed a Multi-modal Feature Fusion Tracking Network for RGB-T(MMFFTN).Firstly,after extracting the preliminary features from the backbone network,the Channel Feature Fusion Module(CFFM)was intro-duced to realize the direct interaction and fusion of RGB and TIR channel features.Secondly,in order to solve the problem of unsatisfactory fusion effect caused by the difference between RGB and TIR modality,a Cross-Modal Feature Fusion Module(CMFM)was designed and the global features of RGB and TIR were further fused through an adaptive fusion strategy to improve the tracking accuracy.The proposed tracking model was evaluated in detail on three datasets:GTOT,RGBT234 and LasHeR.Experimental results demonstrate that MMFFTN improves the success rate and precision rate by 3.0%and 4.7%,re-spectively compared with the current advanced Transformer-based tracker ViPT.Compared with the Transformer-based tracker SDSTrack,the success rate and accuracy are improved by 2.4%and 3.3%,respectively.关键词
RGB-T目标跟踪/Transformer/通道特征融合/跨模态特征融合Key words
RGB-T tracking/transformer/channel feature fusion/cross-modal feature fusion分类
信息技术与安全科学引用本文复制引用
金静,刘建琴,翟凤文..多模态特征融合的RGB-T目标跟踪网络[J].光学精密工程,2025,33(12):1940-1954,15.基金项目
甘肃省高校教师创新基金项目(No.2025B-060) （No.2025B-060）
宁夏自然科学基金资助项目(No.2023AAC03741) （No.2023AAC03741）
甘肃省科技计划项目重点研发计划-工业类(No.23YFGA0047) （No.23YFGA0047）
