智慧农业（中英文）2025，Vol.7Issue(3)：120-130,11.DOI:10.12133/j.smartag.SA202501022
基于改进U-Net模型的高纺锤形苹果树休眠期修剪点识别与定位方法
Pruning Point Recognition and Localization for Spindle-Shaped Apple Trees During Dormant Season Using an Improved U-Net Model
摘要
Abstract
[Objective]To address the current issues of intelligent pruning robots,such as insufficient recognition accuracy of fruit tree branches and inaccurate localization of pruning points in complex field environments,a deep learning method based on the fusion of images and point clouds was proposed in this research.The method enables non-contact segmentation of dormant high-spindle apple tree branches and measurement of phenotypic parameter,which also achieving automatic identification and precise localization of pruning points.[Methods]Localized RGB-D data were collected from apple trees using a Realsense D435i camera,a device capable of accurate depth measurements within the range of 0.3～3.0 m.Data acquisition took place between early and mid-January 2024,from 9:00 AM to 4:00 PM daily.To maintain uniformity,the camera was mounted on a stand at a distance of 0.4～0.5 m from the main stems of the apple trees.Following data collection,trunks and branches were manually annotated using Labelme software.The OpenCV library was also employed for data augmentation,which helped prevent overfitting during model training.To improve seg-mentation accuracy of tree trunks and branches in RGB images,an enhanced U-Net model was introduced.This model utilized VGG16(Visual Geometry Group 16)as its backbone feature extraction network and incorporated the convolutional block attention module(CBAM)at the up-sampling stage.Based on the segmentation results,a multimodal data processing pipeline was established.First,segmented branch mask maps were derived from skeleton lines extracted using OpenCV's algorithm.The first-level branch con-nection points were identified based on their positions relative to the trunk.Subsequently,potential pruning points were then searched within local neighborhoods through coordinate translation.An edge detection algorithm was applied to locate the nearest edge pixels relative to these potential pruning points.By extending the diameter line of branch pixel points in the images and integrating with depth information,the actual diameter of the branches could be estimated.Additionally,branch spacing was calculated using vertical coordinates differences of potential pruning points in the pixel coordinate system,alongside depth information.Meanwhile,trunk point cloud data were acquired by merging the trunk mask maps with the depth maps.Preprocessing of the point cloud enabled the es-timation of the average trunk diameter in the local view through cylindrical fitting using the randomized sampling consistency(RANSAC)algorithm.Finally,an intelligent pruning decision-making algorithm was developed by investigating of orchardists'prun-ing experience,analyzing relevant literature,and integrating phenotypic parameter acquisition methods,thus achieving accurate pre-diction of apple tree pruning points.[Results and Discussions]The improved U-Net model proposed achieved a mean pixel accuracy(mPA)of 95.52%for branch segmentation,representing a 2.74 percent point improvement over the original architecture.Correspond-ing increases were observed in mean intersection over union(mIoU)and precision metrics.Comparative evaluations against Deep-LabV3+,PSPNet,and the baseline U-Net were conducted under both backlight and front-light illumination conditions.The improved model demonstrated superior segmentation performance and robustness across all tested scenarios.Ablation experiments indicated that replacing the original feature extractor with VGG16 yielded a 1.52 percent point mPA improvement,accompanied by simultane-ous gains in mIoU and precision.The integration of the CBAM at the up sampling stage further augmented the model's capacity to re-solve fine branch structures.Phenotypic parameter estimation using segmented branch masks combined with depth maps showed strong correlations with manual measurements.Specifically,the coefficient of determination(R2)values for primary branch diameter,branch spacing,and trunk diameter were 0.96,0.95,and 0.91,respectively.The mean absolute errors(MAE)were recorded as 1.33,13.96,and 5.11 mm,surpassing the accuracy of visual assessments by human pruning operators.The intelligent pruning decision sys-tem achieved an 87.88%correct identification rate for pruning points,with an average processing time of 4.2 s per viewpoint.These results confirm the practical feasibility and operational efficiency of the proposed method in real-world agricultural settings.[Conclu-sions]An efficient and accurate method for identifying pruning points on apple trees was proposed,which integrates image and point cloud data through deep learning.The results indicate that this method could provide significant support for the application of intelli-gent pruning robots in modern agriculture.It not only offers high feasibility but also exhibits outstanding efficiency and accuracy in practical applications,thus laying a solid foundation for the advancement of agricultural automation.关键词
剪枝点识别/RGB-D/U-Net/直径估计/三维点云/VGG16/修剪机器人Key words
pruning point identification/RGB-D/U-Net/diameter estimation/3D point clouds/VGG16/pruning robot分类
信息技术与安全科学引用本文复制引用
刘龙,王宁,王嘉成,曹宇恒,张凯,康峰,王亚雄..基于改进U-Net模型的高纺锤形苹果树休眠期修剪点识别与定位方法[J].智慧农业（中英文）,2025,7(3):120-130,11.基金项目
国家重点研发计划子课题(2018YFD0700603-2) （2018YFD0700603-2）
北京林业大学校院联合基金重点项目(2024XY-G001) （2024XY-G001）
宁夏重点研发计划项目(2022BBF01002-03) The Sub-Project of the National Key Research and Development Program of China(2018YFD0700603-2) （2022BBF01002-03）
Key Project of Beijing Forestry University's University-College Joint Fund(2024XY-G001) （2024XY-G001）
Ningxia Key Research and Development Pro-gram Project(2022BBF01002-03) （2022BBF01002-03）
