Advances in the Applications of Deep Learning Technology for Livestock Smart Farming
Received date: 2022-05-28
Online published: 2022-11-08
Supported by
National Natural Science Foundation Project (62273001)
Accurate and efficient monitoring of animal information, timely analysis of animal physiological and physical health conditions, and automatic feeding and farming management combined with intelligent technologies are of great significance for large-scale livestock farming. Deep learning techniques, with automatic feature extraction and powerful image representation capabilities, solve many visual challenges, and are more suitable for application in monitoring animal information in complex livestock farming environments. In order to further analyze the research and application of artificial intelligence technology in intelligent animal farming, this paper presents the current state of research on deep learning techniques for tag detection recognition, body condition evaluation and weight estimation, and behavior recognition and quantitative analysis for cattle, sheep and pigs. Among them, target detection and recognition is conducive to the construction of electronic archives of individual animals, on which basis the body condition and weight information, behavior information and health status of animals can be related, which is also the trend of intelligent animal farming. At present, intelligent animal farming still faces many problems and challenges, such as the existence of multiple perspectives, multi-scale, multiple scenarios and even small sample size of a certain behavior in data samples, which greatly increases the detection difficulty and the generalization of intelligent technology application. In addition, animal breeding and animal habits are a long-term process. How to accurately monitor the animal health information in real time and effectively feed it back to the producer is also a technical difficulty. According to the actual feeding and management needs of animal farming, the development of intelligent animal farming is prospected and put forward. First, enrich the samples and build a multi perspective dataset, and combine semi supervised or small sample learning methods to improve the generalization ability of in-depth learning models, so as to realize the perception and analysis of the animal's physical environment. Secondly, the unified cooperation and harmonious development of human, intelligent equipment and breeding animals will improve the breeding efficiency and management level as a whole. Third, the deep integration of big data, deep learning technology and animal farming will greatly promote the development of intelligent animal farming. Last, research on the interpretability and security of artificial intelligence technology represented by deep learning model in the breeding field. And other development suggestions to further promote intelligent animal farming. Aiming at the progress of research application of deep learning in livestock smart farming, it provides reference for the modernization and intelligent development of livestock farming.
GUO Yangyang, DU Shuzeng, QIAO Yongliang, LIANG Dong . Advances in the Applications of Deep Learning Technology for Livestock Smart Farming[J]. Smart Agriculture, 2023 , 5(1) : 52 -65 . DOI: 10.12133/j.smartag.SA202205009
1 | QIAO Y L, KONG H, CLARK C, et al. Intelligent perception for cattle monitoring: A review for cattle identification, body condition score evaluation, and weight estimation[J]. Computers and electronics in agriculture, 2021, 185: ID 106143. |
2 | 何东健, 刘冬, 赵凯旋. 精准畜牧业中动物信息智能感知与行为检测研究进展[J]. 农业机械学报, 2016, 47(5): 231-244. |
HE D J, LIU D, ZHAO K X. Review of perceiving animal information and behavior in precision livestock farming[J]. Transactions of the Chinese society for agricultural machinery, 2016, 47(5): 231-244. | |
3 | 秦箐, 刘志红, 赵存, 等. 机器视觉技术在畜牧业中的应用[J]. 农业工程, 2021, 11(7): 27-33. |
QIN Q, LIU Z H, ZHAO C, et al. Application of machine vision technology in livestock and poultry[J]. Agricultural engineering, 2021, 11(7): 27-33. | |
4 | CHEN C, ZHU W X, NORTON T. Behaviour recognition of pigs and cattle: Journey from computer vision to deep learning[J]. Computers and electronics in agriculture, 2021, 187: ID 106255. |
5 | ZIN T T, PHYO C N, TIN P, et al. Image technology based cow identification system using deep learning[C]// International Multi Conference of Engineers and Computer Scientists. Piscataway, NJ, USA: IEEE, 2018 |
6 | 李丹, 张凯锋, 李行健, 等. 基于Mask R-CNN的猪只爬跨行为识别[J]. 农业机械学报, 2019, 50(S1): 261-266, 275. |
LI D, ZHANG K F, LI X J, et al. Mounting behavior recognition for pigs based on mask R-CNN[J]. Transactions of the Chinese society for agricultural machinery, 2019, 50(S1): 261-266, 275. | |
7 | CHENG M, YUAN H B, WANG Q F, et al. Application of deep learning in sheep behaviors recognition and influence analysis of training data characteristics on the recognition effect[J]. Computers and electronics in agriculture, 2022, 198: ID 107010. |
8 | SONG S, LIU T H, WANG H, et al. Using pruning-based YOLOv3 deep learning algorithm for accurate detection of sheep face[J]. Animals: An open access journal from MDPI, 2022, 12(11): ID 1465. |
9 | DENG Z W, VAHDAT A, HU H X, et al. Structure inference machines: Recurrent neural networks for analyzing relations in group activity recognition[C]// IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, USA: IEEE, 2016: 4772-4781. |
10 | KARIM F, MAJUMDAR S, DARABI H, et al. Multivariate LSTM-FCNs for time series classification[J]. Neural networks, 2019, 116: 237-245. |
11 | LI W J, QI F, TANG M, et al. Bidirectional LSTM with self-attention mechanism and multi-channel features for sentiment classification[J]. Neurocomputing, 2020, 387: 63-77. |
12 | LIU J, YANG Y H, LYU S Q, et al. Attention-based BiGRU-CNN for Chinese question classification[J]. Journal of ambient intelligence and humanized computing, 2019: 1-12. |
13 | AWAD A I. From classical methods to animal biometrics: A review on cattle identification and tracking[J]. Computers and electronics in agriculture, 2016, 123: 423-435. |
14 | KAIXUAN Z, DONGJIAN H. Target detection method for moving cows based on background subtraction[J]. International journal of agricultural and biological engineering, 2015, 8(1): 42-49. |
15 | 刘生智, 李春蓉, 刘同金, 等. 基于YOLO V3模型的奶牛目标检测[J]. 塔里木大学学报, 2019, 31(2): 85-90. |
LIU S Z, LI C R, LIU T J, et al. Detection of dairy cows based on YOLOv3 model[J]. Journal of tarim university, 2019, 31(2): 85-90. | |
16 | THARWAT A, GABER T, HASSANIEN A E. Cattle identification based on muzzle images using Gabor features and SVM classifier[C]// International conference on advanced machine learning technologies and applications. Cham, German: Springer, 2014: 236-247. |
17 | KIM H T, IKEDA Y, CHOI H L. The identification of Japanese black cattle by their faces[J]. Asian-australasian journal of animal sciences, 2005, 18(6): 868-872. |
18 | KUMAR S, SINGH S K, SINGH R, et al. Recognition of cattle using face images[M]// Animal biometrics. Singapore: Springer, 2017: 79-110. |
19 | 李向宇, 李慧盈. 基于卷积神经网络的猪脸特征点检测方法[J]. 吉林大学学报(理学版), 2022, 60(3): 609-616. |
LI X Y, LI H Y. Feature point detection method of pig face based on convolutional neural network[J]. Journal of Jilin university (science edition), 2022, 60(3): 609-616. | |
20 | ANDREW W, HANNUNA S, CAMPBELL N, et al. Automatic individual Holstein Friesian cattle identification via selective local coat pattern matching in RGB-D imagery[C]// 2016 IEEE International Conference on Image Processing (ICIP). Piscataway, NJ, USA: IEEE, 2016: 484-488. |
21 | 何屿彤, 李斌, 张锋, 等. 基于改进YOLOv3的猪脸识别[J]. 中国农业大学学报, 2021, 26(3): 53-62. |
HE Y T, LI B, ZHANG F, et al. Pig face recognition based on improved YOLOv3[J]. Journal of China agricultural university, 2021, 26(3): 53-62. | |
22 | 魏斌, MASUM BILLAH, 王美丽, 等. 基于深度学习的羊脸检测与识别方法[J]. 家畜生态学报, 2022, 43(3): 47-50. |
WEI B, BILLAH M, WANG M L, et al. Method of goat face detection and recognition based on deep learning[J]. Journal of domestic animal ecology, 2022, 43(3): 47-50. | |
23 | XUE H, QIN J, QUAN C, et al. Open set sheep face recognition based on euclidean space metric[J]. Mathematical problems in engineering 2021, 2021: 1-5. |
24 | LI X P, DU J Z, YANG J L, et al. When Mobilenetv2 meets transformer: A balanced sheep face recognition model[J]. Agriculture, 2022, 12(8): ID 1126. |
25 | KUMAR S, PANDEY A, SATWIK KSAI RAM, et al. Deep learning framework for recognition of cattle using muzzle point image pattern[J]. Measurement, 2018, 116: 1-17. |
26 | QIAO Y, SU D, KONG H, et al. Individual cattle identification using a deep learning based framework [J]. IFAC-PapersOnLine, 2019, 52(30): 318-323. |
27 | 何东健, 刘建敏, 熊虹婷, 等. 基于改进YOLOv3模型的挤奶奶牛个体识别方法[J]. 农业机械学报, 2020, 51(4): 250-260. |
HE D J, LIU J M, XIONG H T, et al. Individual identification of dairy cows based on improved YOLOv3[J]. Transactions of the Chinese society for agricultural machinery, 2020, 51(4): 250-260. | |
28 | HU H Q, DAI B S, SHEN W Z, et al. Cow identification based on fusion of deep parts features[J]. Biosystems engineering, 2020, 192: 245-256. |
29 | JIANG B, WU Q, YIN X Q, et al. FLYOLOv3 deep learning for key parts of dairy cow body detection[J]. Computers and electronics in agriculture, 2019, 166: ID 104982. |
30 | HODGSON J C, MOTT R, BAYLIS S M, et al. Drones count wildlife more accurately and precisely than humans[J]. Methods in ecology and evolution, 2018, 9(5): 1160-1167. |
31 | ANDREW W, GREATWOOD C, BURGHARDT T. Visual localisation and individual identification of Holstein Friesian cattle via deep learning[C]// 2017 IEEE International Conference on Computer Vision Workshops (ICCVW). Piscataway, NJ, USA: IEEE, 2018: 2850-2859. |
32 | SHAO W, KAWAKAMI R, YOSHIHASHI R, et al. Cattle detection and counting in UAV images based on convolutional neural networks[J]. International journal of remote sensing, 2020, 41(1): 31-52. |
33 | BARBEDO J G A, KOENIGKAN L V, SANTOS P M, et al. Counting cattle in UAV images—Dealing with clustered animals and animal/background contrast changes[J]. Sensors, 2020, 20(7): ID 2126. |
34 | 赵一广, 杨亮, 郑姗姗, 等. 家畜智能养殖设备和饲喂技术应用研究现状与发展趋势[J]. 智慧农业, 2019, 1(1): 20-31. |
ZHAO Y G, YANG L, ZHENG S S, et al. Advances in the development and applications of intelligent equipment and feeding technology for livestock production[J]. Smart agriculture, 2019, 1(1): 20-31. | |
35 | BAHLO C, DAHLHAUS P, THOMPSON H, et al. The role of interoperable data standards in precision livestock farming in extensive livestock systems: A review[J]. Computers and electronics in agriculture, 2019, 156: 459-466. |
36 | CAM M A, OLFAZ M, SOYDAN E. Body measurements reflect body weights and carcass yields in Karayaka sheep[J]. Asian journal of animal and veterinary advances, 2010, 5(2): 120-127. |
37 | DOHMEN R, CATAL C, LIU Q Z. Computer vision-based weight estimation of livestock: A systematic literature review[J]. New Zealand journal of agricultural research, 2022, 65(2/3): 227-247. |
38 | BELL M J, MAAK M, SORLEY M, et al. Comparison of methods for monitoring the body condition of dairy cows[J]. Frontiers in sustainable food systems, 2018, 2: ID 80. |
39 | SALAU J, HAAS J H, JUNGE W, et al. Feasibility of automated body trait determination using the SR4K time-of-flight camera in cow barns[J]. SpringerPlus, 2014, 3: ID 225. |
40 | HALACHMI I, KLOP?I? M, POLAK P, et al. Automatic assessment of dairy cattle body condition score using thermal imaging[J]. Computers and electronics in agriculture, 2013, 99: 35-40. |
41 | LYNN N C, KYU Z M, ZIN T T, et al. Estimating body condition score of cows from images with the newly developed approach[C]// 2017 18th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD). Piscataway, NJ, USA: IEEE, 2017: 91-94. |
42 | 吴宇峰, 李一鸣, 赵远洋, 等. 基于计算机视觉的奶牛体况评分研究综述[J]. 农业机械学报, 2021, 52(S1): 268-275. |
WU Y F, LI Y M, ZHAO Y Y, et al. Review of research on body condition score for dairy cows based on computer vision[J]. Transactions of the Chinese society for agricultural machinery, 2021, 52(S1): 268-275. | |
43 | TEDIN R, BECERRA J A, DURO R J, et al. Towards automatic estimation of the body condition score of dairy cattle using hand-held images and active shape models[M]// Advances in Knowledge-Based and Intelligent Information and Engineering Systems. Amsterdam: IOS Press, 2012: 2150-2159. |
44 | SPOLIANSKY R, EDAN Y, PARMET Y, et al. Development of automatic body condition scoring using a low-cost 3-Dimensional Kinect camera[J]. Journal of dairy science, 2016, 99(9): 7714-7725. |
45 | SUN Y K, HUO P J, WANG Y J, et al. Automatic monitoring system for individual dairy cows based on a deep learning framework that provides identification via body parts and estimation of body condition score[J]. Journal of dairy science, 2019, 102(11): 10140-10151. |
46 | 孔商羽, 陈春雨. 基于多任务学习的猪只体重和体况评分预测[J]. 黑龙江大学工程学报, 2022, 13(2): 70-77. |
KONG S Y, CHEN C Y. Multi-tasking learning on prediction of pig weight and body condition score[J]. Journal of engineering of Heilongjiang university, 2022, 13(2): 70-77. | |
47 | ?EVIK K K, MUSTAFA B. Body condition score (BCS) segmentation and classification in dairy cows using R-CNN deep learning architecture[J]. Avrupa bilim ve teknoloji dergisi, 2019(17): 1248-1255. |
48 | LI X R, HU Z L, HUANG X P, et al. Cow body condition score estimation with convolutional neural networks[C]//2019 IEEE 4th International Conference on Image, Vision and Computing (ICIVC). Piscataway, NJ, USA: IEEE, 2020: 433-437. |
49 | HUANG X P, HU Z L, WANG X R, et al. An improved single shot multibox detector method applied in body condition score for dairy cows[J]. Animals: An open access journal from MDPI, 2019, 9(7): ID 470. |
50 | ALVAREZ J R, ARROQUI M, MANGUDO P, et al. Estimating body condition score in dairy cows from depth images using convolutional neural networks, transfer learning and model ensembling techniques[J]. Agronomy, 2019, 9(2): ID 90. |
51 | DICKINSON R A, MORTON J M, BEGGS D S, et al. An automated walk-over weighing system as a tool for measuring liveweight change in lactating dairy cows[J]. Journal of dairy science, 2013, 96(7): 4477-4486. |
52 | TUYTTENS F A M. The importance of straw for pig and cattle welfare: A review[J]. Applied animal behaviour science, 2005, 92(3): 261-282. |
53 | KASHIHA M, BAHR C, OTT S, et al. Automatic weight estimation of individual pigs using image analysis[J]. Computers and electronics in agriculture, 2014, 107: 38-44. |
54 | HANSEN M F, SMITH M L, SMITH L N, et al. Automated monitoring of dairy cow body condition, mobility and weight using a single 3D video capture device[J]. Computers in industry, 2018, 98: 14-22. |
55 | ZHU A Z, THAKUR D, ?ZASLAN T, et al. The multivehicle stereo event camera dataset: An event camera dataset for 3D perception[J]. IEEE robotics and automation letters, 2018, 3(3): 2032-2039. |
56 | PEZZUOLO A, GUARINO M, SARTORI L, et al. On-barn pig weight estimation based on body measurements by a Kinect v1 depth camera[J]. Computers and electronics in agriculture, 2018, 148: 29-36. |
57 | 张建龙, 冀横溢, 滕光辉. 基于深度卷积网络的育肥猪体重估测[J]. 中国农业大学学报, 2021, 26(8): 111-119. |
ZHANG J L, JI H Y, TENG G H. Weight estimation of fattening pigs based on deep convolutional network[J]. Journal of China agricultural university, 2021, 26(8): 111-119. | |
58 | ZHANG J L, ZHUANG Y R, JI H Y, et al. Pig weight and body size estimation using a multiple output regression convolutional neural network: A fast and fully automatic method[J]. Sensors, 2021, 21(9): ID 3218. |
59 | RUCHAY A, DOROFEEV K, KALSCHIKOV V, et al. Live weight prediction of cattle using deep image regression[C]// 2021 IEEE International Workshop on Metrology for Agriculture and Forestry (MetroAgriFor). Piscataway, NJ, USA: IEEE, 2021: 32-36. |
60 | GJERGJI M, DE MORAES WEBER V, SILVA L O C, et al. Deep learning techniques for beef cattle body weight prediction[C]// 2020 International Joint Conference on Neural Networks (IJCNN). Piscataway, NJ, USA: IEEE, 2020. |
61 | DOHMEN R, CATAL C, LIU Q Z. Image-based body mass prediction of heifers using deep neural networks[J]. Biosystems engineering, 2021, 204: 283-293. |
62 | ACHOUR B, BELKADI M, FILALI I, et al. Image analysis for individual identification and feeding behaviour monitoring of dairy cows based on Convolutional Neural Networks (CNN)[J]. Biosystems engineering, 2020, 198: 31-49. |
63 | ANDRIAMANDROSO A L H, LEBEAU F, BECKERS Y, et al. Development of an open-source algorithm based on inertial measurement units (IMU) of a smartphone to detect cattle grass intake and ruminating behaviors[J]. Computers and electronics in agriculture, 2017, 139: 126-137. |
64 | 王政, 宋怀波, 王云飞, 等. 奶牛运动行为智能监测研究进展与技术趋势[J]. 智慧农业(中英文), 2022, 4(2): 36-52. |
WANG Z, SONG H B, WANG Y F, et al. Research progress and technology trend of intelligent morning of dairy cow motion behavior[J]. Smart agriculture, 2022, 4(2): 36-52. | |
65 | WANG K, WU P, CUI H M, et al. Identification and classification for sheep foraging behavior based on acoustic signal and deep learning[J]. Computers and electronics in agriculture, 2021, 187: ID 106275. |
66 | 应烨伟, 曾松伟, 赵阿勇, 等. 基于颈环采集节点的母羊产前行为识别方法[J]. 农业工程学报, 2020, 36(21): 210-219. |
YING Y W, ZENG S W, ZHAO A Y, et al. Recognition method for prenatal behavior of ewes based on the acquisition nodes of the collar[J]. Transactions of the Chinese society of agricultural engineering, 2020, 36(21): 210-219. | |
67 | 张春慧, 宣传忠, 于文波, 等. 基于三轴加速度传感器的放牧羊只牧食行为研究[J]. 农业机械学报, 2021, 52(10): 307-313. |
ZHANG C H, XUAN C Z, YU W B, et al. Grazing behavior of herding sheep based on three-axis acceleration sensor[J]. Transactions of the Chinese society for agricultural machinery, 2021, 52(10): 307-313. | |
68 | 郝玉胜, 林强, 王维兰, 等. 基于Wi-Fi无线感知技术的奶牛爬跨行为识别[J]. 农业工程学报, 2020, 36(19): 168-176. |
HAO Y S, LIN Q, WANG W L, et al. Recognition of crawling behavior of dairy cows using Wi-Fi wireless sensing technology[J]. Transactions of the Chinese society of agricultural engineering, 2020, 36(19): 168-176. | |
69 | PENG Y Q, KONDO N, FUJIURA T, et al. Classification of multiple cattle behavior patterns using a recurrent neural network with long short-term memory and inertial measurement units[J]. Computers and electronics in agriculture, 2019, 157: 247-253. |
70 | HOSSEININOORBIN S, LAYEGHY S, KUSY B, et al. Deep learning-based cattle behaviour classification using joint time-frequency data representation[J]. Computers and electronics in agriculture, 2021, 187: ID 106241. |
71 | KIM M, CHOI Y, LEE J, et al. A deep learning-based approach for feeding behavior recognition of weanling pigs[J]. Journal of animal science and technology, 2021, 63(6): 1453-1463. |
72 | JIANG M, RAO Y, ZHANG J Y, et al. Automatic behavior recognition of group-housed goats using deep learning[J]. Computers and electronics in agriculture, 2020, 177: ID 105706. |
73 | 王少华, 何东健. 基于改进YOLOv3模型的奶牛发情行为识别研究[J]. 农业机械学报, 2021, 52(7): 141-150. |
WANG S H, HE D J. Estrus behavior recognition of dairy cows based on improved YOLOv3 model[J]. Transactions of the Chinese society for agricultural machinery, 2021, 52(7): 141-150. | |
74 | WU D H, WU Q, YIN X Q, et al. Lameness detection of dairy cows based on the YOLOv3 deep learning algorithm and a relative step size characteristic vector[J]. Biosystems engineering, 2020, 189: 150-163. |
75 | AYADI S, SAID ABEN, JABBAR R, et al. Dairy cow rumination detection: A deep learning approach[C]// In International Workshop on Distributed Computing for Emerging Smart Networks. Berlin,German: Springer, 2020: 123-139. |
76 | FUENTES A, YOON S, PARK J, et al. Deep learning-based hierarchical cattle behavior recognition with spatio-temporal information[J]. Computers and Electronics in Agriculture, 2020, 177: ID 105627. |
77 | CHEN C, ZHU W X, STEIBEL J, et al. Classification of drinking and drinker-playing in pigs by a video-based deep learning method[J]. Biosystems Engineering, 2020, 196: 1-14. |
78 | GUO Y Y, QIAO Y L, SUKKARIEH S, et al. BiGRU-attention based cow behavior classification using video data for precision livestock farming[J]. Transactions of the ASABE, 2021, 64(6): 1823-1833. |
79 | QIAO Y L, GUO Y Y, YU K P, et al. C3D-ConvLSTM based cow behaviour classification using video data for precision livestock farming[J]. Computers and electronics in agriculture, 2022, 193: ID 106650. |
80 | JIANG B, YIN X Q, SONG H B. Single-stream long-term optical flow convolution network for action recognition of lameness dairy cow[J]. Computers and electronics in agriculture, 2020, 175: ID 105536. |
81 | WU D H, WANG Y F, HAN M X, et al. Using a CNN-LSTM for basic behaviors detection of a single dairy cow in a complex environment[J]. Computers and electronics in agriculture, 2021, 182: ID 106016. |
82 | DOUGLAS S L, SZYSZKA O, STODDART K, et al. Animal and management factors influencing grower and finisher pig performance and efficiency in European systems: A meta-analysis[J]. Animal, 2015, 9(7): 1210-1220. |
83 | REICH L J, WEARY D M, VEIRA D M, et al. Effects of sawdust bedding dry matter on lying behavior of dairy cows: A dose-dependent response[J]. Journal of dairy science, 2010, 93(4): 1561-1565. |
84 | DEBRECENI O, LEHOTAYOVá A, BU?KO O, et al. The behaviour of the pigs housed in hot climatic conditions[J]. Journal of central european agriculture, 2014, 15(1): 64-75. |
85 | FREGONESI J A, VEIRA D M, VON KEYSERLINGK M A G, et al. Effects of bedding quality on lying behavior of dairy cows[J]. Journal of dairy science, 2007, 90(12): 5468-5472. |
86 | HEPOLA H, H?NNINEN L, PURSIAINEN P, et al. Feed intake and oral behaviour of dairy calves housed individually or in groups in warm or cold buildings[J]. Livestock science, 2006, 105(1/2/3): 94-104. |
87 | GUO Y Y, HE D J, CHAI L L. A machine vision-based method for monitoring scene-interactive behaviors of dairy calf[J]. Animals: An open access journal from MDPI, 2020, 10(2): ID 190. |
88 | COSTA, ISMAYILOVA, BORGONOVO, et al. Image-processing technique to measure pig activity in response to climatic variation in a pig barn[J]. Animal production science, 2014, 54(8): 1075-1083. |
89 | CHEN C, ZHU W, STEIBEL J, et al. Recognition of feeding behaviour of pigs and determination of feeding time of each pig by a video-based deep learning method[J]. Computers and electronics in agriculture, 2020, 176: ID 105642. |
/
〈 | 〉 |