Automatic Measurement Method of Beef Cattle Body Size Based on Multimodal Image Information and Improved Instance Segmentation Network

doi:10.12133/j.smartag.SA202310007

Abstract

Abstract:

[Objective] The body size parameter of cattle is a key indicator reflecting the physical development of cattle, and is also a key factor in the cattle selection and breeding process. In order to solve the demand of measuring body size of beef cattle in the complex environment of large-scale beef cattle ranch, an image acquisition device and an automatic measurement algorithm of body size were designed. [Methods] Firstly, the walking channel of the beef cattle was established, and when the beef cattle entered the restraining device through the channel, the RGB and depth maps of the image on the right side of the beef cattle were acquired using the Inter RealSense D455 camera. Secondly, in order to avoid the influence of the complex environmental background, an improved instance segmentation network based on Mask2former was proposed, adding CBAM module and CA module, respectively, to improve the model's ability to extract key features from different perspectives, extracting the foreground contour from the 2D image of the cattle, partitioning the contour, and comparing it with other segmentation algorithms, and using curvature calculation and other mathematical methods to find the required body size measurement points. Thirdly, in the processing of 3D data, in order to solve the problem that the pixel point to be measured in the 2D RGB image was null when it was projected to the corresponding pixel coordinates in the depth-valued image, resulting in the inability to calculate the 3D coordinates of the point, a series of processing was performed on the point cloud data, and a suitable point cloud filtering and point cloud segmentation algorithm was selected to effectively retain the point cloud data of the region of the cattle's body to be measured, and then the depth map was 16. Then the depth map was filled with nulls in the field to retain the integrity of the point cloud in the cattle body region, so that the required measurement points could be found and the 2D data could be returned. Finally, an extraction algorithm was designed to combine 2D and 3D data to project the extracted 2D pixel points into a 3D point cloud, and the camera parameters were used to calculate the world coordinates of the projected points, thus automatically calculating the body measurements of the beef cattle. [Results and Discussions] Firstly, in the part of instance segmentation, compared with the classical Mask R-CNN and the recent instance segmentation networks PointRend and Queryinst, the improved network could extract higher precision and smoother foreground images of cattles in terms of segmentation accuracy and segmentation effect, no matter it was for the case of occlusion or for the case of multiple cattles. Secondly, in three-dimensional data processing, the method proposed in the study could effectively extract the three-dimensional data of the target area. Thirdly, the measurement error of body size was analysed, among the four body size measurement parameters, the smallest average relative error was the height of the cross section, which was due to the more prominent position of the cross section, and the different standing positions of the cattle have less influence on the position of the cross section, and the largest average relative error was the pipe circumference, which was due to the influence of the greater overlap of the two front legs, and the higher requirements for the standing position. Finally, automatic body measurements were carried out on 137 beef cattle in the ranch, and the automatic measurements of the four body measurements parameters were compared with the manual measurements, and the results showed that the average relative errors of body height, cross section height, body slant length, and tube girth were 4.32%, 3.71%, 5.58% and 6.25%, respectively, which met the needs of the ranch. The shortcomings were that fewer body-size parameters were measured, and the error of measuring circumference-type body-size parameters was relatively large. Later studies could use a multi-view approach to increase the number of body rule parameters to be measured and improve the accuracy of the parameters in the circumference category. [Conclusions] The article designed an automatic measurement method based on two-dimensional and three-dimensional contactless body measurements of beef cattle. Moreover, the innovatively proposed method of measuring tube girth has higher accuracy and better implementation compared with the current research on body measurements in beef cattle. The relative average errors of the four body tape parameters meet the needs of pasture measurements and provide theoretical and practical guidance for the automatic measurement of body tape in beef cattle.

Key words: cattle body size measurement, deep learning, point cloud segmentation, instance segmentation, attention mechanism, Mask2former

WENG Zhi, FAN Qi, ZHENG Zhiqiang. Automatic Measurement Method of Beef Cattle Body Size Based on Multimodal Image Information and Improved Instance Segmentation Network[J]. Smart Agriculture, 2024, 6(4): 64-75.

Figures/Tables 23

Table 1

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Table 2

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Fig. 9

Fig. 10

Fig. 11

Table 3

Fig. 12

Table 4

Table 5

Fig. 13

Fig. 14

Fig. 15

Fig. 16

Fig. 17

Fig. 18

References 22

1	张智慧, 李伟, 韩永胜. 牛体尺影响因素及其应用[J]. 中国畜牧杂志, 2018, 54(1): 9-13.
	ZHANG Z H, LI W, HAN Y S. Influence factors and applications for body size of cattle[J]. Chinese journal of animal science, 2018, 54(1): 9-13.
2	ZHANG A L, WU B P, TANA WUYUN C, et al. Algorithm of sheep body dimension measurement and its applications based on image analysis[J]. Computers and electronics in agriculture, 2018, 153: 33-45.
3	WANG W, ZHANG Y, HE J, et al. Research on yak body ruler and weight measurement method based on deep learning and binocular vision[EB/OL]. Preprints, 2021, 2021120349. [2023-09-28].
4	GUO H, MA X D, MA Q, et al. LSSA_CAU: An interactive 3d point clouds analysis software for body measurement of livestock with similar forms of cows or pigs[J]. Computers and electronics in agriculture, 2017, 138: 60-68.
5	HUANG L W, GUO H, RAO Q Q, et al. Body dimension measurements of Qinchuan cattle with transfer learning from LiDAR sensing[J]. Sensors, 2019, 19(22): ID 5046.
6	王可, 郭浩, 刘威林, 等. 基于点云旋转归一化的猪体体尺测点提取方法[J]. 农业工程学报, 2017(S1): 253-259.
	WANG K, GUO H, LIU W L, et al. Extraction of porcine body size measurement points based on point cloud rotational normalization[J]. Transactions of the Chinese society of agricultural engineering, 2017(S1): 253-259.
7	张婷, 李琦. 基于三维点云的牛体尺测量研究[J].黑龙江畜牧兽医, 2023, (3): 58-61.
	ZHANG T, LI Q, et al. A study of bovine body measurements based on a three-dimensional point cloud[J]. Heilongjiang animal science and veterinary medicine, 2023, (3): 58-61.
8	LI G X, LIU X L, MA Y F, et al. Body size measurement and live body weight estimation for pigs based on back surface point clouds[J]. Biosystems engineering, 2022, 218: 10-22.
9	YIN L, ZHU J M, LIU C X, et al. Point cloud-based pig body size measurement featured by standard and non-standard postures[J]. Computers and electronics in agriculture, 2022, 199: ID 107135.
10	YANG G Y, XU X S, SONG L, et al. Automated measurement of dairy cows body size via 3D point cloud data analysis[J]. Computers and electronics in agriculture, 2022, 200: ID 107218.
11	石炜, 陈禹州, 张万锴, 等. 基于无人机平台的牛体点云重构及体尺测量系统研究[J]. 畜牧与饲料科学, 2022, 43(4): 93-103.
	SHI W, CHEN Y Z, ZHANG W K, et al. Cattle point cloud reconstruction and body size measurement system development based on unmanned air vehicle(UAV) platform[J]. Animal husbandry and feed science, 2022, 43(4): 93-103.
12	CHENG B W, MISRA I, SCHWING A G, et al. Masked-attention mask transformer for universal image segmentation[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, New Jersey, USA: IEEE, 2022: 1290-1299.
13	王永智. 牛的外貌鉴定、体尺测量及体重估测[J]. 养殖技术顾问, 2013(12): ID 20.
	WANG Y Z. Appearance identification, body measurement and weight estimation of cattle[J]. Technical advisor for animal husbandry, 2013(12): ID 20.
14	CHENG B W, SCHWING A G, KIRILLOV A. Per-pixel classification is not all you need for semantic segmentation[EB/OL]. arXiv: 2107.06278, 2021.
15	ZHU X Z, SU W J, LU L W, et al. Deformable DETR: Deformable transformers for end-to-end object detection[EB/OL]. arXiv: 2010.04159, 2020.
16	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[M]// Computer vision-ECCV 2018. Cham: Springer International Publishing, 2018: 3-19.
17	HOU Q B, ZHOU D Q, FENG J S. Coordinate attention for efficient mobile network design[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, New Jersey, USA: IEEE, 2021: 13713-13722.
18	Abeywickrama T, Aamir Cheema M, Taniar D. k-Nearest Neighbors on Road Networks: A Journey in Experimentation and In-Memory Implementation[EB/OL]. arXiv: 1601.01549, 2016.
19	Bäcklund H, Hedblom A, Neijman N. A density-based spatial clustering of application with noise[J]. Data Mining TNM033, 2011, 33: 11-30.
20	HE K M, GKIOXARI G, DOLLAR P, et al. Mask R-CNN[C]// 2017 IEEE International Conference on Computer Vision (ICCV). Piscataway, New Jersey, USA: IEEE, 2017: 2961-2969.
21	KIRILLOV A, WU Y X, HE K M, et al. PointRend: Image segmentation as rendering[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, New Jersey, USA: IEEE, 2020: 9799-9808.
22	FANG Y X, YANG S S, WANG X G, et al. Instances as queries[C]// 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway, New Jersey, USA: IEEE, 2021: 6910-6919.

模型	bbox_mAP _0.5/%	segm_mAP _0.5/%
Mask2former	83.0	84.3
CB-Mask2former	83.5	84.9
CA-Mask2former	84.5	85.7
CC-Mask2former	86.8	88.1

模型	bbox_mAP _0.5/%	segm_mAP _0.5/%
Mask R-CNN^［20］	84.2	84.2
PointRend^［21］	70.4	69.2
Queryinst^［22］	86.0	84.2
CC-Mask2former	86.8	88.1

[1]	WU Zhangbin, HE Ning, WU Yandong, GUO Xinyu, WEN Weiliang. Point Cloud Data-driven Methods for Estimating Maize Leaf Biomass [J]. Smart Agriculture, 2026, 8(1): 156-166.
[2]	HUANG Xianguo, ZHU Qibing, HUANG Min. Online Detection System for Freshness of Fruits and Vegetables Based on Temporal Multi-source Information Fusion [J]. Smart Agriculture, 2026, 8(1): 203-212.
[3]	HU Yumeng, GUAN Feifan, XIE Dongchen, MA Ping, YU Youben, ZHOU Jie, NIE Yanming, HUANG Lüwen. Tea Leaf Disease Diagnosis Based on Improved Lightweight U-Net3+ [J]. Smart Agriculture, 2026, 8(1): 15-27.
[4]	CAI Yuqin, LIU Daming, XU Qin, LI Boyang, LIU Bojie. Greenhouse Temperature and Humidity Prediction Method Based on Adaptive Kalman Filter and GWO-LSTM-Attention [J]. Smart Agriculture, 2026, 8(1): 148-155.
[5]	YAO Xiaotong, QU Shaoye. Lightweight Detection Method for Pepper Leaf Diseases and Pests Based on Improved YOLOv12s [J]. Smart Agriculture, 2026, 8(1): 1-14.
[6]	ZHANG Yun, ZHANG Lumin, XU Guangtao, HAO Jiahui. Remote Sensing Extraction Method of Rice-Crayfish Fields Based on Dual-Branch and Multi-Scale Attention [J]. Smart Agriculture, 2025, 7(6): 185-195.
[7]	ZHAO Jun, NIE Zhigang, LI Guang, LIU Jiayu. Corn Borer Pests Infestations Detection Method Using Low-Altitude Close-Range UAV Imagery [J]. Smart Agriculture, 2025, 7(6): 111-123.
[8]	LI Wenzheng, YANG Xinting, SUN Chuanheng, CUI Tengpeng, WANG Hui, LI Shanshan, LI Wenyong. Light-Trapping Rice Planthopper Detection Method by Combining Spatial Depth Transform Convolution and Multi-scale Attention Mechanism [J]. Smart Agriculture, 2025, 7(5): 169-181.
[9]	HAN Wenkai, LI Tao, FENG Qingchun, CHEN Liping. Lightweight Apple Instance Segmentation Algorithm Based on SSW-YOLOv11n for Complex Orchard Environments [J]. Smart Agriculture, 2025, 7(5): 114-123.
[10]	WANG Fengyun, WANG Xuanyu, AN Lei, FENG Wenjie. Detection Method for Log-Cultivated Shiitake Mushrooms Based on Improved RT-DETR [J]. Smart Agriculture, 2025, 7(5): 67-77.
[11]	ZHAO Yingping, LIANG Jinming, CHEN Beizhang, DENG Xiaoling, ZHANG Yi, XIONG Zheng, PAN Ming, MENG Xiangbao. Applications Research Progress and Prospects of Multi-Agent Large Language Models in Agricultural [J]. Smart Agriculture, 2025, 7(5): 37-51.
[12]	LIU Yiheng, LIU Libo. Beef Cattle Object Detection Method Under Occlusion Environment Based on Improved YOLOv12 [J]. Smart Agriculture, 2025, 7(5): 182-192.
[13]	HU Yan, WANG Yujie, ZHANG Xuechen, ZHANG Yiqiang, YU Huahao, SONG Xinbei, YE Sitan, ZHOU Jihong, CHEN Zhenlin, ZONG Weiwei, HE Yong, LI Xiaoli. Non-Destructive Inspection and Intelligent Grading Method of Fu Brick Tea at Fungal Fermentation Stage Based on Hyperspectral Imaging Technology [J]. Smart Agriculture, 2025, 7(4): 71-83.
[14]	WANG Yi, XUE Rong, HAN Wenting, SHAO Guomin, HOU Yanqiao, CUI Xitong. Estimation of Maize Aboveground Biomass Based on CNN-LSTM-SA [J]. Smart Agriculture, 2025, 7(4): 159-173.
[15]	LI Ruijie, WANG Aidong, WU Huaxing, LI Ziqiu, FENG Xiangqian, HONG Weiyuan, TANG Xuejun, QIN Jinhua, WANG Danying, CHU Guang, ZHANG Yunbo, CHEN Song. Remote Sensing for Rice Growth Stages Monitoring: Research Progress, Bottleneck Problems and Technical Optimization Paths [J]. Smart Agriculture, 2025, 7(3): 89-107.