高电压技术2025，Vol.51Issue(2)：642-651,10.DOI:10.13336/j.1003-6520.hve.20232230
基于多模态的缺陷绝缘子图像的多标签分类
Multi-label Classification of Defective Insulator Images Based on Multimodality
摘要
Abstract
Accurate classification of insulator defects in inspection images is one of the key technologies in the field of automatic inspection of transmission lines.To address the issue of the insufficient utilization of textual information by traditional deep learning classification methods and the issue of relatively simplistic insulator image classification labels,this paper proposes for the first time a multi-label classification method for defective insulator images based on a multi-modal approach.Firstly,a multimodal joint data augmentation method is employed,achieving cross-modal data enhancement between insulator images and label texts.Then,the Vision Transformer network is utilized to extract fea-tures from images,and the BERT network is used to extract features from label texts,fully leveraging the feature information from both images and label texts to obtain comprehensive information from different modalities,thereby en-hancing the network's classification capabilities.Finally,through correlating the feature information of images and texts via contrastive learning,the reliability of network classification is enhanced,while also providing good interpretability for the classification results.The experimental results demonstrate that this method achieves an overall accuracy rate of 93.87%,showing a significant advantage in classification performance over other models on the same dataset.关键词
绝缘子图像/多标签分类/多模态/对比学习/数据增强Key words
insulator images/multi-label classification/multi-modal/contrastive learning/data augmentation引用本文复制引用
周景,王满意,田兆星..基于多模态的缺陷绝缘子图像的多标签分类[J].高电压技术,2025,51(2):642-651,10.基金项目
国家电网公司科技项目(5108-202218280A-2-400-XG).Project supported by Science and Technology Project of SGCC(5108-202218280A-2-400-XG). （5108-202218280A-2-400-XG）
