基于样本动态权重的课程式半监督学习方法OA北大核心CSTPCD
Curriculum paradigm based on the dynamic weights of samples for semi-supervised learning
本文针对半监督场景中极度匮乏的监督信号导致的标签传播困难、模型训练严重受噪声干扰等问题展开研究.伪标签化带来的噪声和低数据利用率导致的确认偏差,会随着自训练过程造成错误累积,进而形成不可逆偏差,损害性能.本文提出基于样本动态权重的课程式半监督学习方法,旨在通过非离散的课程设计,鼓励模型由简单至困难地利用样本,逐步构建分类面,进而缓解伪标签化过程中的噪声产生,增强模型泛化能力.从类内角度,提供弱监督信号的高置信度伪标签被混合用于构建特征原型,估计样本的学习难度.从类间角度,标签嵌入被用于评估类间语义相关度,课程式地减弱训练前期对语义相关类别间的辨别.在通用的半监督学习基准数据集上进行了广泛的实验和分析,证明了方法的有效性.
This work studies the difficulty of label propagation and serious noise interference in model training,which are due to the extreme lack of supervision signals in semi-supervised learning scenarios.Noise from pseudo-labeling and confirmation bias caused by low data utilization will lead to error accumulation along with the self-training process,thus forming irreversible deviation and damaging the performance.In this paper,a curriculum paradigm based on the dynamic weights of samples for semi-supervised learning is proposed,aiming at encouraging the model to utilize samples from easy to hard and gradually construct hyperplanes based on the non-discrete curriculum,so as to alleviate the generation of noise in the pseudo-labeling process and enhance the generalization ability of the mod-el.Specifically,from the intra-class perspective,prototypes of features are constructed by mixing pseudo-labels with high confidence,which can provide weak supervision signals.Then,the learning difficulties of samples are es-timated.From the inter-class perspective,label embedding is used to evaluate the semantic relevancy between cate-gories,and the discrimination between semantically related categories are weaken in the early stage of training.Comprehensive experiments and analyses are conducted on commonly-used semi-supervised learning benchmark datasets to demonstrate the effectiveness of this method.
朱徽;胡斌;宋怡宁;赵晓芳
中国科学院计算技术研究所 北京 100190||中国科学院大学 北京 100049中国科学院计算技术研究所 北京 100190中央军委国防动员部信息中心 北京 100034中国科学院计算技术研究所 北京 100190||中科苏州智能计算技术研究院 苏州 215028
半监督学习特征表示向量课程学习特征原型语义相关度
semi-supervised learningfeature representation vectorcurriculum learningprototype of fea-turessemantic relevancy
《高技术通讯》 2024 (004)
342-355 / 14
国家重点研发计划(2021YFF0703800)资助项目.
评论