欢迎您访问《智慧农业(中英文)》官方网站! English

Smart Agriculture ›› 2024, Vol. 6 ›› Issue (4): 76-90.doi: 10.12133/j.smartag.SA202401023

• 专题--智慧畜牧技术创新与可持续发展 • 上一篇    下一篇

猪三维点云体尺自动计算模型Pig Back Transformer

王宇啸2, 石源源1, 陈招达1, 吴珍芳2,3,4, 蔡更元2,3,4, 张素敏1, 尹令1,2,4()   

  1. 1. 华南农业大学 数学与信息学院,广东 广州 510642,中国
    2. 国家生猪种业工程技术研究中心,广东 广州 510642,中国
    3. 华南农业大学 动物科学学院,广东 广州 510642,中国
    4. 猪禽种业全国重点实验室,广东 广州 510640,中国
  • 收稿日期:2024-01-21 出版日期:2024-07-30
  • 基金项目:
    国家自然科学基金面上基金(32172780); 国家重点研发计划子课题(2023YFD1300202)
  • 作者简介:
    王宇啸,研究方向为计算机视觉。E-mail:
  • 通信作者:
    尹 令,博士,教授,研究方向为三维视觉。E-mail:

Pig Back Transformer: Automatic 3D Pig Body Measurement Model

WANG Yuxiao2, SHI Yuanyuan1, CHEN Zhaoda1, WU Zhenfang2,3,4, CAI Gengyuan2,3,4, ZHANG Sumin1, YIN Ling1,2,4()   

  1. 1. College of Mathematics and Informatics, South China Agricultural University, Guangzhou 510642, China
    2. National Engineering Research Center For Breeding Swine Industry, Guangzhou 510642, China
    3. College of Animal Science, South China Agricultural University, Guangzhou 510642, China
    4. National Key Laboratory of Pig and Poultry Breeding Industry, Guangzhou 510640, China
  • Received:2024-01-21 Online:2024-07-30
  • Foundation items:General Program of the National Natural Science Foundation of China(32172780); Sub-Project of the National Key R&D Program(2023YFD1300202)
  • About author:
  • Corresponding author:
    YIN Ling, E-mail:

摘要:

[目的/意义] 为了提高体尺关键点定位准确率,猪三维点云体尺自动测量方法会采用点云分割,在各个分割后局部点云定位测量关键点,以减少点云之间相互干扰。然而点云分割网络通常需要消耗较大计算资源,且现有测量点定位效果仍有待提升空间。本研究旨在通过设计关键点生成网络从猪体点云中提取出各体尺测量所需关键点。在降低显存资源需求的同时提高测量关键点定位效果,提高体尺测量的效率和精度。 [方法] 针对猪三维表面点云进行体尺测量,提出了一种定位猪体尺关键点的模型Pig Back Transformer。模型分为两个模块,分别设计了两种改进的Transformer自注意力编码器,第一模块为全局关键点模块,首先设计了一种猪背部边缘点提取算法用于获取边缘点,再使用edge encoder编码器以边缘点集合作为输入,edge encoder的edge attention中加入了边缘点和质点的偏移距离信息;第二模块为关键点生成模块,使用了back attention机制的back encoder,其中加入了与质心和第一模块生成的全局关键点的偏移量,并将偏移量与点云注意力通过按位max pooling操作结合,最后通过生成猪的体尺测量关键点和背脊走向点。最后设计了使用关键点和背脊走向点作为输入的体尺算法。 [结果和讨论] 对比关键点和背脊走向点生成任务上Pig Back Transformer表现最佳,并对比体尺计算结果与人工测量结果,体长相对误差为0.63%,相对PointNet++、Point Transformer V2、Point Cloud Transforme、OctFormer PointTr等模型有较大提升。 [结论] Pig Back Transformer能相对准确地生成猪体尺关键点,提高体尺测量数据准确度,并且通过点云特征定位体尺关键点节省了计算资源,为无接触牲畜体尺测量提供了新思路。

关键词: Pig Back Transformer, 三维点云, 体尺自动测量, 测量关键点定位, 深度相机, 自注意力机制

Abstract:

[Objective] Nowadays most no contact body size measurement studies are based on point cloud segmentation method, they use a trained point cloud segmentation neural network to segment point cloud of pigs, then locate measurement points based on them. But point cloud segmentation neural network always need a larger graphics processing unit (GPU) memory, moreover, the result of the measurement key point still has room of improvement. This study aims to design a key point generating neural network to extract measurement key points from pig's point cloud. Reducing the GPU memory usage and improve the result of measurement points at the same time, improve both the efficiency and accuracy of the body size measurement. [Methods] A neural network model was proposed using improved Transformer attention mechanic called Pig Back Transformer for generating key points and back orientation points which were related to pig body dimensions. In the first part of the network, it was introduced an embedding structure for initial feature extraction and a Transformer encoder structure with edge attention which was a self-attention mechanic improved from Transformer's encoder. The embedding structure using two shared multilayer perceptron (MLP) and a distance embedding algorithm, it takes a set of points from the edge of pig back's point cloud as input and then extract information from the edge points set. In the encoder part, information about the offset distances between edge points and mass point which were their feature that extracted by the embedding structure mentioned before incorporated. Additionally, an extraction algorithm for back edge point was designed for extracting edge points to generate the input of the neural network model. In the second part of the network, it was proposed a Transformer encoder with improved self-attention called back attention. In the design of back attention, it also had an embedding structure before the encoder structure, this embedding structure extracted features from offset values, these offset values were calculated by the points which are none-edge and down sampled by farthest point sampling (FPS) to both the relative centroid point and model generated global key point from the first part that introduced before. Then these offset values were processed with max pooling with attention generated by the extracted features of the points' axis to extract more information that the original Transformer encoder couldn't extract with the same number of parameters. The output part of the model was designed to generate a set of offsets of the key points and points for back direction fitting, than add the set offset to the global key point to get points for pig body measurements. At last, it was introduced the methods for calculating body dimensions which were length, height, shoulder width, abdomen width, hip width, chest circumference and abdomen circumference using key points and back direction fitting points. [Results and Discussions] In the task of generating key points and points for back direction fitting, the improved Pig Back Transformer performed the best in the accuracy wise in the models tested with the same size of parameters, and the back orientation points generated by the model were evenly distributed which was a good preparation for a better body length calculation. A melting test for edge detection part with two attention mechanic and edge trim method both introduced above had being done, when the edge detection and the attention mechanic got cut off, the result had been highly impact, it made the model couldn't perform as well as before, when the edge trim method of preprocessing part had been cut off, there's a moderate impact on the trained model, but it made the loss of the model more inconsistence while training than before. When comparing the body measurement algorithm with human handy results, the relative error in length was 0.63%, which was an improvement compared to other models. On the other hand, the relative error of shoulder width, abdomen width and hip width had edged other models a little but there was no significant improvement so the performance of these measurement accuracy could be considered negligible, the relative error of chest circumference and abdomen circumference were a little bit behind by the other methods existed, it's because the calculate method of circumferences were not complicated enough to cover the edge case in the dataset which were those point cloud that have big holes in the bottom of abdomen and chest, it impacted the result a lot. [Conclusions] The improved Pig Back Transformer demonstrates higher accuracy in generating key points and is more resource-efficient, enabling the calculation of more accurate pig body measurements. And provides a new perspective for non-contact livestock body size measurements.

Key words: Pig Back Transformer, 3D point cloud, body size automic measurement, key point positioning, depth camera, self-attention mechanism