农业工程学报2019，Vol.35Issue(18)：135-143,9.DOI:10.11975/j.issn.1002-6819.2019.18.017
基于相位相关的温室番茄植株多模态三维重建方法
Multi-modal three-dimensional reconstruction of greenhouse tomato plants based on phase-correlation method
摘要
Abstract
In order to realize plant morphological measurement and physiological diagnosis, a multi-modal three-dimensional (3D) reconstruction method was proposed. This reconstruction method further laid the foundation for plant phenotypic measurement. Due to the complexity of 3D geometric morphologies, only two or two-and-a-half-dimensional images of greenhouse plants can be captured at a single angle of view (AOV) by the imaging sensors. However, 3D point cloud reconstruction of plants requires images captured at multiple AOVs. In addition, the 3D geometric morphologies of plants undergo significant changes during the full-growth-cycle and to acquire suitable 3D plant images, it is necessary to frequently adjust the sensor position. Therefore, sensor position and AOV directly affect the plant phenotyping efficiency. Developing an efficient and accurate multi-view 3D point cloud reconstruction method that meets the need for full-growth-cycle, high-throughput 3D reconstruction and phenotyping of greenhouse plants is therefore pivotal to the development of high-throughput plant phenotyping techniques. So a multi-modal three-dimensional reconstruction method of greenhouse tomato plants under different measurement positions and angles was proposed, and to solve the problem of multi-spectral reflectance mapping and multi-view point cloud 3D reconstruction, multi-spectral reflectance images were registered to RGB-D image coordinate system by phase-correlation method, and a multi-view RGB-D image 3D reconstruction method based on self-calibration of the Kinect sensor was established which realized the reconstruction of RGB 3D point cloud model and multi-spectral reflectance 3D point cloud model of the plants. The two-dimensional multi-spectral image registration quality was evaluated objectively by the normalizing gray-scale similarity coefficient D, the spectral overlap rate in the region of interest (ROI) C, and the mutual information value and the Hausdorff distance HD was applied to objectively evaluate the reconstruction accuracy of the three-dimensional point cloud reconstruction of the plant. In total, 30 greenhouse tomato plants were used in this study with each plant reconstructed from four angles of view at angle intervals of 90 degrees. The obtained results showed that the average values of C and D were 0.920 6 and 0.908 5, respectively. After registration, the mutual information value increased by 9.81 % and the canopy multi-spectral images could be registered accurately to the depth coordinate system. The ratio of the HD distance set of reconstruction point cloud less than 0.6 cm was 78.39 %, the ratio of less than 1.0 cm was 91.13%, and the mean value of the tomato distance ensemble HDavg was 0.37 cm, depicting that the tomato plant 3D point cloud model had high reconstruction accuracy and could be applied to multi-modal 3D reconstruction of greenhouse tomato plants. This research integrated the traditional 3D geometric morphology measurement system and plant physiological information diagnosis system, and as such the external morphology and internal physiological information of the plants could be measured in the same imaging room. It provides a precise and efficient measurement method for high-throughput plant phenotypic measurement, and is of great significance to the development of modern intelligent facility horticultural management and plant phenomics.关键词
图像重构/三维/立体视觉/相位相关/多模态/多光谱/三维重建/温室番茄/植物表型Key words
image reconstruction/ three dimensional/ stereo vision/ phase correlation/ multi-modal/ multi-spectral/three-dimensional reconstruction/ greenhouse tomato/ plant phenotypic分类
信息技术与安全科学引用本文复制引用
孙国祥,汪小旵,刘景娜,孙晔,丁永前,卢伟..基于相位相关的温室番茄植株多模态三维重建方法[J].农业工程学报,2019,35(18):135-143,9.基金项目
江苏省自然科学基金资助项目(BK20170727) （BK20170727）
中央高校基本科研业务费专项资金资助(KYGX201703) （KYGX201703）
江苏省自然科学基金资助项目(BK20181315) （BK20181315）
