农业工程学报2017，Vol.33Issue(10)：59-69,11.DOI:10.11975/j.issn.1002-6819.2017.10.008
苹果采摘机器人视觉系统研究进展
Research advance on vision system of apple picking robot
摘要
Abstract
Vision system is one of the most important parts of apple picking robot, which, to some extent, determines the quality and the speed of picking task implemented by apple picking robot. In this review, we enumerated the existing apple picking robots. Some information, such as type of visual sensors, hardware of vision system, success rate of harvesting and run time, was illustrated in details. Meanwhile, on the basis of discussing the vision system of apple picking robot, we focused on summarizing hardware structure of existing vision systems and apple image segmentation methods as well as apple recognition and localization methods applied in vision systems. The vision system of apple picking robot mainly includes machine vision system, laser vision system, three-dimensional vision system and vision system formed by machine vision and other vision system. And the machine vision system can be classified into 3 types according to the number of image sensors used, that is, monocular vision system, binocular vision system and multi vision system. The recognition and localization of apple target is the first step of the implementation of the picking task for picking robot. The currently used apple segmentation and recognition methods include threshold segmentation algorithm, chromatic aberration based algorithm, K-mean clustering algorithm, region growing algorithm, segmentation method combining 2 or more algorithms, and so on. There are 4 methods that are commonly used in the localization of apple target. They are the methods based on centroid, fitting circle, symmetry axes, and three-dimensional coordinates, respectively. In natural scene, the recognition and localization of apple target may be affected by many factors. Hence, recognition and localization of apple target under different conditions, such as color nonuniformity, different illumination, shadow on the surface, oscillation, overlapping and occlusion, was reviewed and analyzed. Among all these conditions, occlusion can be regarded as the most serious factor. The condition of occlusion can be roughly divided into 4 kinds, i.e. apple target blocked by other apple, by branches, by leaves and by branches, leaves and other apple simultaneously. As for apple targets blocked by branches, one apple may be separated by branches, thus causing that an apple may be recognized as several apples. For the apple targets blocked by leaves, the symmetry of apple can be utilized to localize apple targets. The apple targets blocked by other apple can be considered as overlapping. There are overlapped apples with series connection, parallel connection, and blend connection. Because of the complexity of overlapping, the recognition and localization of apple targets blocked by other apple target is a little more difficult. In addition, the detection of obstacles like tree trunk and branches in apple orchard is important for apple picking robots to avoid obstacles, and thus obstacles detection methods were summarized in this review. In the process of target recognition and localization, binocular vision technology was commonly used in vision system. The key point of binocular vision technology is stereo matching. Therefore, stereo matching was then reviewed, and the image matching methods in existence can generally be divided into 2 categories, i.e. region-based image matching method and feature-based image matching method. What's more, the problems exist in recognition and localization methods used in the vision system of apple picking robot, including accuracy, effectiveness, character of real-time and universal applicability, were analyzed. Further study will concentrate on optimizing the structure of vision system, optimizing the intelligent algorithms used in the vision system, improving the real-time capability, recognizing and locating apple targets when the apple targets and vision systems are influenced by oscillation, and improving cost performance. The paper has summarized and analyzed vision system of apple picking robot comprehensively, which can provide reference for future research.关键词
机器人/图像识别/机械化/苹果/果实识别/目标定位/视觉系统Key words
robots/image recognition/mechanization/apple/fruit recognition/target localization/vision system分类
信息技术与安全科学引用本文复制引用
王丹丹,宋怀波,何东健..苹果采摘机器人视觉系统研究进展[J].农业工程学报,2017,33(10):59-69,11.基金项目
陕西省自然科学基金资助(2014JQ3094) （2014JQ3094）
陕西省农业科技创新与攻关项目(2016NY-157) （2016NY-157）
中央高校基本科研业务经费(2452016077). （2452016077）
