基于多种深度学习算法的田间玉米籽粒检测与计数

doi:10.12133/j.smartag.SA202207004

Smart Agriculture ›› 2022, Vol. 4 ›› Issue (4): 49-60.doi: 10.12133/j.smartag.SA202207004

• 专题--大田作物智慧种植 • 上一篇下一篇

基于多种深度学习算法的田间玉米籽粒检测与计数

刘晓航^1,²(), 张昭^1,²(), 刘嘉滢^1,², 张漫^1,², 李寒^1,², FLORES Paulo³, 韩雄哲^4,⁵

^1.中国农业大学智慧农业系统集成研究教育部重点实验室，北京 100083
^2.中国农业大学农业农村部农业信息获取技术重点实验室，北京 100083
^3.北达科他州州立大学农业与生物工程系，法戈 58102，美国
^4.韩国江原大学生物系统工程系，春川 24341，韩国
^5.韩国江原大学智慧农业交叉学科，春川 24341，韩国

收稿日期:2022-07-12 出版日期:2022-12-30
基金资助:
中央高校基本科研业务费专项资金资助(00112502);世界顶尖涉农大学国际合作交流种子基金(15052001)
作者简介:刘晓航（1996－），男，博士研究生，研究方向为智慧农业。E-mail：lxhhaust@163.com
通信作者:

Infield Corn Kernel Detection and Counting Based on Multiple Deep Learning Networks

LIU Xiaohang^1,²(), ZHANG Zhao^1,²(), LIU Jiaying^1,², ZHANG Man^1,², LI Han^1,², FLORES Paulo³, HAN Xiongzhe^4,⁵

^1.Key Laboratory of Smart Agriculture System Integration, Ministry of Education, China Agricultural University, Beijing 100083, China
^2.Key Lab of Agricultural Information Acquisition Technology, Ministry of Agriculture and Rural Affairs, China Agricultural University, Beijing 100083, China
^3.Department of Agricultural and Biosystems Engineering, North Dakota State University, Fargo, ND 58012, United States
^4.Department of Biosystems Engineering, Kangwon National University, Chuncheon 24341, Korea
^5.Interdisciplinary Program in Smart Agriculture, Kangwon National University, Chuncheon 24341, Korea

Received:2022-07-12 Online:2022-12-30

摘要/Abstract

摘要：

为快速准确获取玉米收获过程中遗失籽粒数信息，进行收割损失调节等管理，对比评估了单阶段和两阶段主流目标检测网络对田间玉米籽粒计数的性能。首先，利用RGB相机获取包含不同背景和不同光照的图像数据，并进一步生成数据集；其次，构建籽粒识别的不同目标检测网络，包括Mask R-CNN、EfficientDet-D5、YOLOv5-L、YOLOX-L，并利用所采集的420幅有效图像对构建的四种网络进行训练、验证、测试，图像数分别为200、40和180幅；最后，依据测试集图像的识别结果进行籽粒计数性能评价。试验结果表明，YOLOv5-L网络对测试集图像检测的平均精度为78.3%，模型尺寸仅为89.3 MB；籽粒计数的检测正确率、漏检率和F₁值分别为90.7%、9.3%和91.1%，处理速度为55.55 f/s，识别与计数性能均优于Mask R-CNN、EfficientDet-D5和YOLOX-L网络，并对具有不同地表遮挡程度和籽粒聚集状态的图像具有较强的鲁棒性。深度学习目标检测网络YOLOv5-L可实现实际作业中玉米收获损失籽粒的实时监测，精度高、适用性强。

关键词: 收获损失, 田间玉米籽粒, 深度学习, 籽粒计数, YOLOv5-L, YOLOX-L, Mask R-CNN, EfficientDet-D5

Abstract:

Machine vision has been increasingly used for agricultural sensing tasks. The detection method based on deep learning for infield corn kernels can improve the detection accuracy. In order to obtain the number of lost corn kernels quickly and accurately after the corn harvest, and evaluate the corn harvest combine performance on grain loss, the method of directly using deep learning technology to count corn kernels in the field was developed and evaluated. Firstly, an RGB camera was used to collect image with different backgrounds and illuminations, and the datasets were generated. Secondly, different target detection networks for kernel recognition were constructed, including Mask R-CNN, EfficientDet-D5, YOLOv5-L and YOLOX-L, and the collected 420 effective images were used to train, verify and test each model. The number of images in train, verify and test datasets were 200, 40 and 180, respectively. Finally, the counting performances of different models were evaluated and compared according to the recognition results of test set images. The experimental results showed that among the four models, YOLOv5-L had overall the best performance, and could reliably identify corn kernels under different scenes and light conditions. The average precision (AP) value of the model for the image detection of the test set was 78.3%, and the size of the model was 89.3 MB. The correct rate of kernel count detection in four scenes of non-occlusion, surface mid-level-occlusion, surface severe-occlusion and aggregation were 98.2%, 95.5%, 76.1% and 83.3%, respectively, and F₁ values were 94.7%, 93.8%, 82.8% and 87%, respectively. The overall detection correct rate and F₁ value of the test set were 90.7% and 91.1%, respectively. The frame rate was 55.55 f/s, and the detection and counting performance were better than Mask R-CNN, EfficientDet-D5 and YOLOX-L networks. The detection accuracy was improved by about 5% compared with the second best performance of Mask R-CNN. With good precision, high throughput, and proven generalization, YOLOv5-L can realize real-time monitoring of corn harvest loss in practical operation.

Key words: harvest loss, infield corn kernel, deep learning, kernel count, YOLOv5-L, YOLOX-L, Mask R-CNN, EfficientDet-D5

中图分类号:

S126

刘晓航, 张昭, 刘嘉滢, 张漫, 李寒, FLORES Paulo, 韩雄哲. 基于多种深度学习算法的田间玉米籽粒检测与计数[J]. 智慧农业(中英文), 2022, 4(4): 49-60.

LIU Xiaohang, ZHANG Zhao, LIU Jiaying, ZHANG Man, LI Han, FLORES Paulo, HAN Xiongzhe. Infield Corn Kernel Detection and Counting Based on Multiple Deep Learning Networks[J]. Smart Agriculture, 2022, 4(4): 49-60.

参考文献

1	张伟锋, 刘新娇, 李秋枫, 等. 世界玉米生产与贸易概况分析[J]. 世界农业, 2014(3): 111-114.
1	ZHANG W, LIU X, LI Q, et al. General situation analysis of world corn production and trade[J]. World Agriculture, 2014(3): 111-114.
2	WU Y, LI X, MAO E, et al. Design and development of monitoring device for corn grain cleaning loss based on piezoelectric effect[J]. Computers and Electronics in Agriculture, 2020, 179(12): ID 105793.
3	XU L, WEI C, LIANG Z, et al. Development of rapeseed cleaning loss monitoring system and experiments in a combine harvester[J]. Biosystems Engineering, 2019, 178: 118-130.
4	VALIENTE-GONZáLEZJM, ANDREU-GARCíAG, POTTER P, et al. Automatic corn (Zea mays) kernel inspection system using novelty detection based on principal component analysis[J]. Biosystems Engineering, 2014, 117: 94-103.
5	ORLANDI G, CALVINI R, FOCA G, et al. Automated quantification of defective maize kernels by means of multivariate image analysis[J]. Food Control, 2018, 85: 259-268.
6	LI X, DAI B, HONG S, et al. Corn classification system based on computer vision[J]. Symmetry, 2019, 11(4): 591-591.
7	张恬, 赵德安, 周童. 图像处理在联合收割机夹带损失检测中的应用[J]. 农机化研究, 2009, 31(4): 70-72.
7	ZHANG T, ZHAO D, ZHOU T. Application of image processing on combine harvester attachment loss[J]. Journal of Agricultural Mechanization Research, 2009, 31(4): 70-72.
8	辛博, 吴涛, 陈春林, 等. 一种基于图像处理的谷物收割清选损失实时在线检测方法: CN107123115A[P]. 2017-09-01.
8	XIN B, WU T, CHEN C, et al. A real-time online detection method for grain harvesting and cleaning loss based on image processing: CN107123115A[P]. 2017-09-01.
9	WELLINGTON C K, BRUNS A J, SIERRA V S, et al. Grain quality monitoring: US10664726B2[P]. 2017-10-02.
10	孙红, 李松, 李民赞, 等. 农业信息成像感知与深度学习应用研究进展[J]. 农业机械学报, 2020, 51(5): 1-17.
10	SUN H, LI S, LI M, et al. Research progress of image sensing and deep learning in agriculture[J]. Transactions of the CSAM, 2020, 51(5): 1-17.
11	MONHOLLEN N S, SHINNERS K J, FRIEDE J C, et al. In-field machine vision system for identifying corn kernel losses[J]. Computers and Electronics in Agriculture, 2020, 174: ID 105496.
12	吕璐, 程虎, 朱鸿泰, 等. 基于深度学习的目标检测研究与应用综述[J]. 电子与封装, 2022, 22(1): 72-80.
12	LYU L, CHENG H, ZHU H, et al. Progress of research and application of object detection based on deep learning[J]. Electronics & Packaging, 2022, 22(1): 72-80.
13	董丽君, 曾志高, 易胜秋, 等. 基于YOLOv5的遥感图像目标检测[J]. 湖南工业大学学报, 2022, 36(3): 44-50.
13	DONG L, ZENG Z, YI S, et al. Research on a YOLOv5-Based remote sensing image target detection[J]. Journal of Hunan University of Technology, 2022, 36(3): 44-50.
14	谢富, 朱定局. 深度学习目标检测方法综述[J]. 计算机系统应用, 2022, 31(2): 1-12.
14	XIE F, ZHU D. Survey on deep learning object detection[J]. Computer Systems & Applications, 2022, 31(2): 1-12.
15	寇大磊, 权冀川, 张仲伟. 基于深度学习的目标检测框架进展研究[J]. 计算机工程与应用, 2019, 55(11): 25-34.
15	KOU D, QUAN J, ZHANG Z. Research on progress of object detection framework based on deep learning[J]. Computer Engineering and Applications, 2019, 55(11): 25-34.
16	包晓敏, 王思琪. 基于深度学习的目标检测算法综述[J]. 传感器与微系统, 2022, 41(4): 5-9.
16	BAO X, WANG S. Survey of object detection algorithm based on deep learning[J]. Transducer and Microsystem Technologies, 2022, 41(4): 5-9.
17	HE K, GKIOXARI G, DOLLáR P, et al. Mask r-cnn[C]// The IEEE International Conference on Computer Vision. Piscataway, New York, USA: IEEE, 2017: 2961-2969.
18	TAN M, PANG R, LE Q. Efficientdet: Scalable and efficient object detection[C]// The IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, New York, USA: IEEE, 2020: 10781-10790.
19	BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: Optimal speed and accuracy of object detection[J/OL]. arXiv:2004.10934[cs.CV], 2020.
20	GE Z, LIU S, WANG F, et al. YOLOx: Exceeding yolo series in 2021[J/OL]. arXiv:, 2021.

[1]	毛克彪, 张晨阳, 施建成, 王旭明, 郭中华, 李春树, 董立新, 吴门新, 孙瑞静, 武胜利, 姬大彬, 蒋玲梅, 赵天杰, 邱玉宝, 杜永明, 徐同仁. 基于人工智能的地球物理参数反演范式理论及判定条件[J]. 智慧农业(中英文), 2023, 5(2): 161-171.
[2]	夏雪, 柴秀娟, 张凝, 周硕, 孙琦鑫, 孙坦. 用于边缘计算设备的果树挂果量轻量化估测模型[J]. 智慧农业(中英文), 2023, 5(2): 1-12.
[3]	郭阳阳, 杜书增, 乔永亮, 梁栋. 深度学习在家畜智慧养殖中研究应用进展[J]. 智慧农业(中英文), 2023, 5(1): 52-65.
[4]	左敏, 胡天宇, 董微, 张可心, 张青川. 基于Informer神经网络的农产品物流需求预测分析——以华中地区为例[J]. 智慧农业(中英文), 2023, 5(1): 34-43.
[5]	计洁, 金洲, 王儒敬, 刘海燕, 李志远. 基于递进式卷积网络的农业命名实体识别方法[J]. 智慧农业(中英文), 2023, 5(1): 122-131.
[6]	胡松涛, 翟瑞芳, 王应华, 刘志, 朱剑忠, 任荷, 杨万能, 宋鹏. 基于多源数据的马铃薯植株表型参数提取[J]. 智慧农业(中英文), 2023, 5(1): 132-145.
[7]	许钰林, 康孟珍, 王秀娟, 华净, 王浩宇, 沈震. 基于深度学习的玉米和大豆期货价格智能预测[J]. 智慧农业(中英文), 2022, 4(4): 156-163.
[8]	罗庆, 饶元, 金秀, 江朝晖, 王坦, 王丰仪, 张武. 基于改进YOLOv5s和多模态图像的树上毛桃检测[J]. 智慧农业(中英文), 2022, 4(4): 84-104.
[9]	张志博, 赵西宁, 高晓东, 张利, 杨孟豪. 基于改进Linknet网络的黄土高原苹果园精准提取[J]. 智慧农业(中英文), 2022, 4(3): 95-107.
[10]	商枫楠, 周学成, 梁英凯, 肖明玮, 陈桥, 罗陈迪. 基于改进YOLOX的自然环境中火龙果检测方法[J]. 智慧农业(中英文), 2022, 4(3): 120-131.
[11]	何锐敏, 郑可锋, 尉钦洋, 张小斌, 张俊, 朱怡航, 赵懿滢, 顾清. 基于改进Mask R-CNN模型的工厂化养蚕蚕体识别与计数[J]. 智慧农业(中英文), 2022, 4(2): 163-173.
[12]	庄家煜, 许世卫, 李杨, 熊露, 刘克宝, 钟志平. 基于深度学习的多种农产品供需预测模型[J]. 智慧农业(中英文), 2022, 4(2): 174-182.
[13]	邵明月, 张建华, 冯全, 柴秀娟, 张凝, 张文蓉. 深度学习在植物叶部病害检测与识别的研究进展[J]. 智慧农业(中英文), 2022, 4(1): 29-46.
[14]	杨国峰, 何勇, 冯旭萍, 李禧尧, 张金诺, 俞泽宇. 无人机遥感监测作物病虫害胁迫方法与最新研究进展[J]. 智慧农业(中英文), 2022, 4(1): 1-16.
[15]	王莹, 李越, 武婷婷, 孙石, 王敏娟. 基于密度估计和VGG-Two的大豆籽粒快速计数方法[J]. 智慧农业(中英文), 2021, 3(4): 111-122.

基于多种深度学习算法的田间玉米籽粒检测与计数

Infield Corn Kernel Detection and Counting Based on Multiple Deep Learning Networks

在线阅读

知网下载

本地下载

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价