电子学报2025，Vol.53Issue(3)：716-727,12.DOI:10.12263/DZXB.20240357
基于伪标签去噪和SAM优化的大规模无监督语义分割
Pseudo-label Denoising and SAM Optimization for Large-scale Unsupervised Semantic Segmentation
摘要
Abstract
Semantic segmentation technology enables fine-grained understanding of complex and diverse scenes and is one of the key technologies to promote efficient and intelligent work of unmanned systems.Large-scale unsupervised se-mantic segmentation aims to learn semantic segmentation capabilities from a large number of unlabeled images.However,the existing approaches suffer heavily from their noisy self-learned pseudo-labels with poor category and shape representa-tions,leading to low final segmentation accuracy.In this paper,we propose a Pseudo-label Denoising and SAM Optimiza-tion(PDSO)approach for large-scale unsupervised semantic segmentation to alleviate the problem mentioned above.Specif-ically,we first propose a denoising-based feature fine-tuning module,which fine-tunes the pre-trained backbone network with clean image-level pseudo-label samples selected from a large dataset based on a small loss criterion,enabling the net-work to obtain more robust category representations.To further reduce category noise in pseudo-labels,we propose a clus-tering-based sample denoising module to discard noisy samples that interfere with clustering based on the category propor-tion and the distances between samples and cluster centers,thereby enhancing clustering performance.Moreover,we pro-pose a SAM prompt optimization module,which identifies active categories in the image based on clustering distance to fil-ter out noisy targets and uses points and boxes as SAM's target prompt information to generate expected target masks and refine the edges of targets in pseudo-labels.Our proposed PDSO reaches the mIoU of 45.0%,26.6%,and 14.5%on the test set of ImageNet-S50,ImageNet-S300,and ImageNet-S919 datasets,respectively,which significantly improves the category accuracy and edge accuracy of the segmented targets.关键词
大规模无监督语义分割/图像级去噪/分割一切模型/伪标签/聚类Key words
large-scale unsupervised semantic segmentation/image-level denoising/segment anything model/pseu-do-label/clustering分类
计算机与自动化引用本文复制引用
杨维静,徐瑞,顾浩文,陈涛,舒祥波,姚亚洲..基于伪标签去噪和SAM优化的大规模无监督语义分割[J].电子学报,2025,53(3):716-727,12.基金项目
国家自然科学基金(No.62302217) （No.62302217）
装备发展部信息系统共用技术预研项目(No.31511030202) National Natural Science Foundation of China(No.62302217) （No.31511030202）
Information Systems Common Technology Pre-research Project of Equipment Development Department(No.31511030202) （No.31511030202）
