自动化与信息工程2026，Vol.47Issue(4)：8-16,9.DOI:10.12475/aie.20260402
多模态蒸馏驱动的单RGB鲁棒目标检测方法
Multi-modal Distillation-driven Robust Object Detection Method with Single RGB
摘要
Abstract
To address the insufficient robustness of lightweight single-RGB object detection models under complex illumination,low visibility,or partial occlusion conditions,as well as the challenges of high hardware cost,complex calibration,and limited power consumption faced by multimodal sensor solutions in edge deployment,this paper proposes multi-modal distillation-driven robust object detection method with single RGB.The method constructs a heterogeneous teacher-student distillation framework in which a multimodal teacher network guides the training phase while only the single-RGB student model is retained for inference.During training,the teacher network is built with RGB images and auxiliary modal images,transferring structural information from thermal infrared,depth,and other auxiliary modalities to a lightweight student network that receives only RGB inputs through degradation modeling,cross-modal semantic projection,cross-modal consistency constraints,and modal-compensation attention mechanisms.During inference,only the single-RGB student network is retained to enable low-cost deployment.To enhance experimental verifiability,comparative experiments,missing-modality generalization experiments,ablation studies,and deployment experiments are conducted on the self-built multimodal training dataset USAB-180K and the public benchmark KAIST dataset.Experimental results demonstrate that the proposed method achieves a 3.4 percentage point improvement in Macro-F 1 over the single-RGB baseline model on the KAIST test set,reduces the false positive rate by 5.6 percentage points,and increases mAP@50 from 62.4％to 72.1％.Under 1080p,30 f/s video stream input,when deployed on the RV1106 industrial vision platform,the model achieves an average end-to-end latency of 31.8 ms and an inference power consumption of 206 mW.The proposed method provides a feasible approach for enhancing the stability of object detection performance on resource-constrained edge devices in complex environments.关键词
目标检测/多模态蒸馏/单RGB视觉/知识蒸馏/边缘部署Key words
object detection/multi-modal distillation/single-RGB vision/knowledge distillation/edge deployment分类
信息技术与安全科学引用本文复制引用
艾炎,叶廷东..多模态蒸馏驱动的单RGB鲁棒目标检测方法[J].自动化与信息工程,2026,47(4):8-16,9.基金项目
广东省普通高校重点科研平台和项目(2023ZDZX1053) （2023ZDZX1053）
