高电压技术2025，Vol.51Issue(2)：630-641,12.DOI:10.13336/j.1003-6520.hve.20232123
基于多模态对比学习的输电线路螺栓缺陷分类
Transmission Line Bolt Defects Classification Based on Multi-modal Contrastive Learning
摘要
Abstract
Bolt images collected in transmission line inspection have the characteristics of low resolution and insufficient visual information.To solve the problem that traditional image classification models struggle to learn rich-semantic visual representations from bolt images,this paper proposes a method of bolt defect classification based on multi-modal contras-tive learning.Firstly,in order to inject bolt-related semantic information and prior knowledge into the visual representation in a cross-modal manner,a two-stage training algorithm which combines the multi-modal contrastive pre-training and supervised fine-tuning is proposed.Secondly,to alleviate the overfitting in multi-modal contrastive pre-training,the info noise contrastive estimation loss with label smoothing(infoNCE-LS)is proposed to improve the generalization of the pre-trained visual representation.Finally,aimed at the mismatch between the upstream and down-stream tasks,three types of classification heads based on text prompts are designed to improve the transfer learning performance of the pre-trained visual representation in the supervised fine-tuning stage.The experimental results show that the accuracy of two models based on ResNet50 and ViT on the bolt defect classification dataset is 92.3%and 97.4%,which is 2.4%and 5.8%higher than the baseline.The study realizes the cross-modal supplement of semantic information from text to image,which provides a new idea for the research of bolt defect identification.关键词
输电线路/螺栓缺陷分类/多模态预训练/对比学习/迁移学习Key words
transmission line/bolt defect classification/multi-modal pre-training/contrastive learning/transfer learning引用本文复制引用
张珂,郑朝烨,石超君,赵振兵,肖扬杰..基于多模态对比学习的输电线路螺栓缺陷分类[J].高电压技术,2025,51(2):630-641,12.基金项目
国家自然科学基金(62076093 （）
62206095 （）
61871182) （）
中央高校基本科研业务费专项资金(2023JG002 （）
2022MS078 （）
2023JC006).Project supported by National Natural Science Foundation of China(62076093,62206095,61871182),Fundamental Research Funds for the Central Universities(2023JG002,2022MS078,2023JC006). （62076093,62206095,61871182）
