光学精密工程2024，Vol.32Issue(7)：1087-1100,14.DOI:10.37188/OPE.20243207.1087
CNN-Transformer结合对比学习的高光谱与LiDAR数据协同分类
Collaborative classification of hyperspectral and LiDAR data based on CNN-transformer
摘要
Abstract
To tackle the challenges in multimodal classification tasks involving hyperspectral images(HSI)and LiDAR data,such as cross-modal information expression and feature alignment,this paper introduces a contrastive learning-based multi-branch CNN-Transformer network(CLCT-Net)for the joint classifica-tion of hyperspectral and LiDAR data.Initially,CLCT-Net employs a feature extraction module with a ConvNeXt V2 Block to capture shared features across different modalities,addressing the semantic align-ment issue between data from heterogeneous sensors.It then develops a dual-branch HSI encoder with spa-tial channel and spectral context branches,alongside a LiDAR encoder enhanced by a frequency domain self-attention mechanism,to secure more comprehensive feature representations.Lastly,it leverages en-semble contrastive learning for classification to further refine the accuracy of multimodal collaborative clas-sification.Experimental evaluations on the Houston 2013 and Trento datasets demonstrate that the pro-posed model excels in extracting and integrating cross-modal data features,achieving superior ground ob-ject classification accuracies of 92.01%and 98.90%,respectively,when compared to existing models for classifying hyperspectral images and LiDAR data.关键词
高光谱图像/激光雷达数据/Transformer/卷积神经网络/对比学习Key words
hyperspectral image/LiDAR data/transformer/convolutional neural network/contrastive learning分类
信息技术与安全科学引用本文复制引用
吴海滨,戴诗语,王爱丽,岩堀祐之,于效宇..CNN-Transformer结合对比学习的高光谱与LiDAR数据协同分类[J].光学精密工程,2024,32(7):1087-1100,14.基金项目
黑龙江省自然科学基金资助项目(No.JJ2023LH1143) （No.JJ2023LH1143）
黑龙江省重点研发计划资助项目(No.JD2023SJ19) （No.JD2023SJ19）
"一带一路"创新人才交流外国专家项目(No.G2022012010L) （No.G2022012010L）
黑龙江省级领军人才梯队后备带头人资助项目 （）
