光学精密工程2025，Vol.33Issue(6)：945-960,16.DOI:10.37188/OPE.20253306.0945
基于策略梯度和伪孪生网络的异源图像匹配
Cross-modality image matching algorithm based on policy gradient and pseudo-twin network
摘要
Abstract
A multi-source image matching method named PCMM-Net was proposed to address the prob-lem of unmatched keypoints resulting from the different imaging mechanisms of visible and infrared imag-es.Firstly,a U-Net model with a policy gradient mechanism was introduced as the baseline model to ex-tract keypoints from the images.This foundational model transformed pixel values into normalized proba-bilities,serving to filter out low-texture areas.This process enabled the network to focus on and learn key-points that were both reliable and repeatable.Then,to address the radiance discrepancies between visible images and infrared images,a pseudo-twin network was employed to extract similar features from local im-age patches.Finally,a fusion layer was proposed to integrate similar features and features from keypoint detectors,generating descriptors suitable for multi-source image matching.The proposed algorithm was validated for matching performance on the VEDAI near-infrared dataset and the MTV thermal infrared da-taset.Experimental results demonstrate that the proposed algorithm achieves average matching accuracies of 97.77%and 95.88%on the VEDAI and MTV datasets,respectively.Compared to the DALF algo-rithm,the average matching accuracies are improved by 2.26%and 14%on VEDAI and MTV datasets.Experimental results show that the algorithm has better matching effect and improves the accuracy of matching.关键词
可见光与红外图像/异源图像匹配/深度学习/卷积神经网络Key words
visible and infrared images/multi-source image matching/deep learning/Convolutional Neural Network(CNN)分类
计算机与自动化引用本文复制引用
张荐,梁奥,花海洋,刘天赐,李峙含..基于策略梯度和伪孪生网络的异源图像匹配[J].光学精密工程,2025,33(6):945-960,16.基金项目
国家博士后面上基金(No.2020M681006) （No.2020M681006）
