智慧农业（中英文）2024，Vol.6Issue(4)：64-75,12.DOI:10.12133/j.smartag.SA202310007
基于多模态图像信息及改进实例分割网络的肉牛体尺自动测量方法
Automatic Measurement Method of Beef Cattle Body Size Based on Multimodal Image Information and Improved Instance Segmentation Network
摘要
Abstract
[Objective]The body size parameter of cattle is a key indicator reflecting the physical development of cattle,and is also a key factor in the cattle selection and breeding process.In order to solve the demand of measuring body size of beef cattle in the complex environ-ment of large-scale beef cattle ranch,an image acquisition device and an automatic measurement algorithm of body size were designed. [Methods]Firstly,the walking channel of the beef cattle was established,and when the beef cattle entered the restraining device through the channel,the RGB and depth maps of the image on the right side of the beef cattle were acquired using the Inter RealSense D455 camera.Secondly,in order to avoid the influence of the complex environmental background,an improved instance segmenta-tion network based on Mask2former was proposed,adding CBAM module and CA module,respectively,to improve the model's abili-ty to extract key features from different perspectives,extracting the foreground contour from the 2D image of the cattle,partitioning the contour,and comparing it with other segmentation algorithms,and using curvature calculation and other mathematical methods to find the required body size measurement points.Thirdly,in the processing of 3D data,in order to solve the problem that the pixel point to be measured in the 2D RGB image was null when it was projected to the corresponding pixel coordinates in the depth-valued image,resulting in the inability to calculate the 3D coordinates of the point,a series of processing was performed on the point cloud data,and a suitable point cloud filtering and point cloud segmentation algorithm was selected to effectively retain the point cloud data of the region of the cattle's body to be measured,and then the depth map was 16.Then the depth map was filled with nulls in the field to retain the integrity of the point cloud in the cattle body region,so that the required measurement points could be found and the 2D data could be returned.Finally,an extraction algorithm was designed to combine 2D and 3D data to project the extracted 2D pixel points into a 3D point cloud,and the camera parameters were used to calculate the world coordinates of the projected points,thus auto-matically calculating the body measurements of the beef cattle. [Results and Discussions]Firstly,in the part of instance segmentation,compared with the classical Mask R-CNN and the recent in-stance segmentation networks PointRend and Queryinst,the improved network could extract higher precision and smoother fore-ground images of cattles in terms of segmentation accuracy and segmentation effect,no matter it was for the case of occlusion or for the case of multiple cattles.Secondly,in three-dimensional data processing,the method proposed in the study could effectively extract the three-dimensional data of the target area.Thirdly,the measurement error of body size was analysed,among the four body size mea-surement parameters,the smallest average relative error was the height of the cross section,which was due to the more prominent po-sition of the cross section,and the different standing positions of the cattle have less influence on the position of the cross section,and the largest average relative error was the pipe circumference,which was due to the influence of the greater overlap of the two front legs,and the higher requirements for the standing position.Finally,automatic body measurements were carried out on 137 beef cattle in the ranch,and the automatic measurements of the four body measurements parameters were compared with the manual measure-ments,and the results showed that the average relative errors of body height,cross section height,body slant length,and tube girth were 4.32%,3.71%,5.58%and 6.25%,respectively,which met the needs of the ranch.The shortcomings were that fewer body-size parameters were measured,and the error of measuring circumference-type body-size parameters was relatively large.Later studies could use a multi-view approach to increase the number of body rule parameters to be measured and improve the accuracy of the pa-rameters in the circumference category. [Conclusions]The article designed an automatic measurement method based on two-dimensional and three-dimensional contactless body measurements of beef cattle.Moreover,the innovatively proposed method of measuring tube girth has higher accuracy and bet-ter implementation compared with the current research on body measurements in beef cattle.The relative average errors of the four body tape parameters meet the needs of pasture measurements and provide theoretical and practical guidance for the automatic mea-surement of body tape in beef cattle.关键词
肉牛体尺测量/深度学习/点云分割/实例分割/注意力机制/Mask2formerKey words
cattle body size measurement/deep learning/point cloud segmentation/instance segmentation/attention mechanism/Mask2former分类
信息技术与安全科学引用本文复制引用
翁智,范琦,郑志强..基于多模态图像信息及改进实例分割网络的肉牛体尺自动测量方法[J].智慧农业（中英文）,2024,6(4):64-75,12.基金项目
国家自然科学基金项目(61966026) （61966026）
内蒙古自治区高等学校青年科技英才支持计划(NJYT23063) （NJYT23063）
内蒙古自然科学基金项目(2021MS06014) National Natural Science Foundation of China(61966026) （2021MS06014）
Program For Young Talents of Science And Technology In Universities of Inner Mongolia Autonomous Region Grant(NJYT23063) （NJYT23063）
Natural Science Foundation of Inner Mongolia under Grant(2021MS06014) （2021MS06014）
