基于YOLOv5s微光环境下的多模态识别网络OA
Multimodal Recognition Network in Low Light Environment Based on YOLOv5s
近些年来,在目标检测以及图像分割等领域涌现了许多先进的算法.在能见度较差的微光场景下如夜晚、大雾天气等场景中,视频图像具有像素声高、对比度低、无彩色信息等特点,算法的检测性能受到明显限制.与目前主流的RGB相机相比,毫米波雷达对上述复杂环境具有一定的免疫能力,可以在不利条件下辅助RGB相机进行目标检测工作.以单阶段目标检测器中实时性较高的YOLOv5s为基础,结合毫米波雷达的特性,提出了用于微光环境下目标检测的多模态识别网络.与现有的传感器融合方法相比,多模态识别网络有几个关键优势.系统以基于学习的方式融合了2种模态,只需要少量新场景的标记图像和雷达数据,因为其可以充分利用已经开源的大型图像数据集进行大批量的训练.这一突出特性使新系统能够适应高度复杂的现实环境.由于使用了高度计算效率的融合方法,系统是非常轻量级的,因此适用于各个复杂场景下的实时应用.为了评估系统的性能,制作了一个小批量的雷达和摄像机融合数据集,包含普通光照和不同强度微光光照条件下的多模态数据.实验结果表明,微光场景下多模态识别网络的平均精度达到76.6％,相比Faster R-CNN算法和YOLOv7算法,全类平均精度(mean Average Precision,mAP)提高了 16.8％和9.3％,且误检、漏检率低,达到了在微光环境下完成目标检测任务的要求.
In recent years,many advanced algorithms have emerged in the fields of object detection and image segmentation.However,in low light scenarios with poor visibility,such as night and foggy weather,video images have the characteristics of high pixel noise,low contrast and no color information,which significantly limits the detection performance of the algorithm.Meanwhile,compared with mainstream RGB cameras,the millimeter wave radar has certain immunity to the complex environment mentioned above,and can assist RGB cameras in object detection under adverse conditions.Therefore,based on YOLOv5s,which has high real-time performance in single stage object detectors,and combined with the characteristics of millimeter wave radar,a multimodal recognition network is proposed for object detection in low light environments.Compared with existing sensor fusion methods,the proposed multimodal recognition network has several key advantages.Firstly,although this system integrates two modalities in a learning based manner,it only requires a small amount of labeled images and radar data for new scenes,as it can fully utilize large open source image datasets for large-scale training.This outstanding feature enables the new system to adapt to highly complex real-world environments.Secondly,due to the use of highly computationally efficient fusion methods,this system is very lightweight and therefore suitable for real-time applications in various complex scenarios.In order to evaluate the performance of this system,a small batch of radar and camera fusion dataset is produced,which includes multimodal data under normal lighting and different intensities of low light illumination.The experimental results show that the average accuracy of the multimodal recognition network in low light environments reaches 76.6％.Compared with Faster R-CNN algorithm and YOLOv7 algorithm,the mean Average Precision(mAP)has improved by 16.8％and 9.3％,and the false detection and missed detection rates are low,meeting the requirements for object recognition in low light environment.
吴学礼;赵俊棋;刘雨涵;甄然
河北科技大学电气工程学院,河北石家庄 050018||河北省生产过程自动化工程技术研究中心,河北石家庄 050018河北科技大学电气工程学院,河北石家庄 050018
计算机与自动化
多模态识别网络毫米波雷达目标检测YOLOv5s微光场景
multimodal recognition networkmillimeter wave radarobject detectionYOLOv5slow light scene
《无线电工程》 2024 (007)
1602-1613 / 12
国家自然科学基金(62003129);河北省重点研发计划项目(19250801D)National Natural Science Foundation of China(62003129);Key R&D Plan Projects in Hebei Province(19250801D)
评论