农业工程学报2025，Vol.41Issue(12)：77-85,9.DOI:10.11975/j.issn.1002-6819.202502099
基于RGB-D图像的高纺锤形苹果树修剪执行末端位姿估计
Estimating end-effector pose for pruning tall-spindle apple trees using RGB-D images
摘要
Abstract
Pruning is one of the most critical steps in the cultivation of fruit trees.Current pruning robots have realized to recognize the side branch,and then locate the pruning points in recent years.However,it is still lacking in the effective end-effector pose estimation in intelligent selective pruning.This study aims to propose the pruning point localization and end-effector pose estimation using RGB-D images.The research object was also selected as the dormant high spindle-shaped apple trees.A depth camera(Intel RealSense D435i)was utilized to capture the RGB and depth data.A point-to-plane mapping was introduced to derive the 3D orientation and position of the pruning pose from the detected pixel coordinates and depth information.The spatial location was predicted for the cutting plane's orientation relative to the pruning point—a key requirement for autonomous robotic pruning.In the perception pipeline,an improved version of the YOLOv8-seg model was employed to segment the trunk and primary branch regions from the RGB images.Furthermore,it was often lacking on the clear boundary features of the branch base masks,due to the unconventional annotation.The original YOLOv8-seg model failed to accurately locate and then segment these regions.A Global Attention Mechanism(GAM)module was introduced into the neck network of YOLOv8-seg.Each C2f block was then integrated across all feature levels.The feature maps were also recalibrated using channel-wise multiplication,in order to enhance the salient features while suppressing the irrelevant ones.The multi-scale information and reasoning were significantly enhanced for the high accuracy of the segmentation.The improved YOLOv8-seg model was achieved in a mask-level precision of 95.31%,recall of 93.79%,and an mAP0.5 of 93.86%,thus outperforming the original YOLOv8-seg by 0.79 percentage points in precision,2.63 percentage points in recall,and 1.47 percentage points in mAP0.5.Once the trunk and primary branches were segmented,the OpenCV-based image processing was applied to calculate the diameters and spacing of the branches.The potential pruning points were identified to fit the rectangles around the base regions of the side branches,according to the empirical pruning.Field trials were carried out to validate the effectiveness of this approach.A better performance was achieved,with a decision accuracy of 88.3%and an average processing speed of 2.1 seconds per image.Extensive testing showed that the point-to-plane mapping of the pose estimation was achieved with a success rate of 89.9%,with an average computation time of 3.3 s per image.In conclusion,a framework was presented for the intelligent selective pruning of the apple trees using RGB-D input,in order to realize the accurate pruning point localization and end-effector pose estimation.Advanced deep learning models were also integrated with the image processing.The pruning pose can be expected to align with the specific angles for the tree's health.The point-to-plane mapping can be expected to determine the spatial location of the pruning points.The optimal orientation of the cutting plane can also be calculated to fully meet the horticultural requirements of the pruning actions.Specifically,the normal vector of the cutting plane was derived,according to the detected pruning points and surrounding branch structures.The manipulator's reachability and safety distances can be considered to generate feasible pruning poses for practical execution.The pruning end-effector pose estimation can also provide strong support for developing robotic pruning.关键词
图像处理/实例分割/RGB-D/YOLOv8-seg/苹果树/修剪/位姿估计Key words
image processing/instance segmentation/RGB-D/YOLOv8-seg/apple trees/pruning/pose estimation分类
农业科技引用本文复制引用
康峰,王嘉成,王亚雄,王宁..基于RGB-D图像的高纺锤形苹果树修剪执行末端位姿估计[J].农业工程学报,2025,41(12):77-85,9.基金项目
国家重点研发计划子课题项目(2018YFD0700603-2) （2018YFD0700603-2）
宁夏揭榜挂帅项目(2022BBF01002) （2022BBF01002）
