南京农业大学学报2025，Vol.48Issue(2)：476-487,12.DOI:10.7685/jnau.202401007
复杂背景下草莓点云语义分割优化方法
Optimization method for semantic segmentation of strawberry point cloud under complex background
摘要
Abstract
[Objectives]In view of the difficulties such as interference from field background noise,the small size of strawberry fruits and the presence of occlusion,this paper utilizes 3D vision technology to achieve accurate recognition and positioning of strawberries,so as to provide technical support for the automatic picking by robots.[Methods]The Intel Realsense D435i depth camera was used to collect strawberry point cloud data under different lighting,different seasons and occlusion conditions,and a dataset containing three categories was constructed,namely non-occluded,low-occluded and high-occluded.Combined with multi-threshold statistical filtering and region of interest(ROI)extraction techniques,the point cloud data were preprocessed to filter out noise.Taking PointNet++as the basic model,features were directly extracted from the point cloud data,and on the basis of PointNet++,a semantic segmentation model SS-PointNet++for small-scale targets in complex backgrounds was proposed.Multiple feature information of the point cloud was used as the network input features to construct sampling layers and grouping layers,and local features were extracted through PointNet.The farthest point sampling method was used to sample the point cloud and cover the entire point set to the greatest extent.For small-scale targets,three ball queries with different radii were designed to obtain local features,and the structures of the set abstraction(SA)layer and the feature propagation(FP)layer were improved to make them adaptable to low-density point clouds.[Results]When segmenting the point cloud without preprocessing,there was a 0.74%probability of misjudgment of outliers.Meanwhile,the average time for semantic segmentation of a single preprocessed point cloud image was reduced by 3.47 seconds.The test results of point cloud images showed that the average accuracy rate of the SS-PointNet++model reached 86.95%,an increase of 19.54 percentage points compared with that before optimization,and the average intersection over union was 0.740.On strawberries with sufficient light and no occlusion,the semantic segmentation accuracy rate of this model was as high as 95.36%,while in a low-light environment,the average accuracy rate of this model could also reach 81.34%.[Conclusions]The SS-PointNet++model had improved the semantic segmentation effect of small-scale target point clouds and had strong robustness under different lighting conditions,providing an effective method for the segmentation of small objects and occluded objects based on 3D point clouds.The classification of strawberry occlusion types proposed in this paper provided data analysis support for subsequent strawberry occlusion problems and also served as a reference for target detection and occlusion problems of other small-scale objects based on 3D point clouds.关键词
草莓/点云/采摘机器人/计算机视觉/语义分割/深度相机Key words
strawberry/point cloud/picking robot/computer vision/semantic segmentation/depth camera分类
信息技术与安全科学引用本文复制引用
谢元澄,陈自强,许忠义,严心悦,姜海燕,梁敬东..复杂背景下草莓点云语义分割优化方法[J].南京农业大学学报,2025,48(2):476-487,12.基金项目
国家自然科学基金项目(31872847) （31872847）
