电子学报2024，Vol.52Issue(3)：696-708,13.DOI:10.12263/DZXB.20220593
一种多层多模态融合3D目标检测方法
3D Object Detection Based on Multilayer Multimodal Fusion
摘要
Abstract
Camera and lidar are the key sources of information in autonomous vehicles(AVs).However,in the cur-rent 3D object detection tasks,most of the pure point cloud network detection capabilities are better than those of image and laser point cloud fusion networks.Existing studies summarize the reasons for this as the misalignment of view between im-age and radar information and the difficulty of matching heterogeneous features.Single-stage fusion algorithm is difficult to fully fuse the features of both.For this reason,a nova 3D object detection based on multilayer multimodal fusion(3DMMF)is presented.First,in the early-fusion phase,point clouds are encoded locally by Frustum-RGB-PointPainting(FRP)formed by the 2D detection frame.Then,the encoded point cloud input is combined with the self-attention mechanism context-aware channel to expand the PointPillars detection network.In the later-fusion phase,2D and 3D candidate boxes are coded as two sets of sparse tensors before they are not greatly suppressed,and the final 3D target detection result is obtained by us-ing the camera lidar object candidates fusion(CLOCs)network.Experiments on KITTI datasets show that this fusion detec-tion method has a significant performance improvement over the baseline of pure point cloud networks,with an average mAP improvement of 6.24%.关键词
自动驾驶/多传感器融合/3D目标检测/点云编码/自注意力机制Key words
auto-driving/multi-sensor fusion/3D target detection/point cloud coding/self-attention mechanism分类
信息技术与安全科学引用本文复制引用
周治国,马文浩..一种多层多模态融合3D目标检测方法[J].电子学报,2024,52(3):696-708,13.基金项目
装备预研领域基金(No.61403120109) Equipment Pre-Research Field Foundation(No.61403120109) （No.61403120109）
