Yield Estimation Method of Apple Tree Based on Improved Lightweight YOLOv5

doi:10.12133/j.smartag.2021.3.2.202105-SA005

Abstract

Abstract:

Yield estimation of fruit tree is one of the important works in orchard management. In order to improve the accuracy of in-situ yield estimation of apple trees in orchard, a method for the yield estimation of single apple tree, which includes an improved YOLOv5 fruit detection network and a yield fitting network was proposed. The in-situ images of the apples without bags at different periods were acquired by using an unmanned aerial vehicle and Raspberry Pi camera, formed an image sample data set. For dealing with no attention preference and the parameter redundancy in feature extraction, the YOLOv5 network was improved by two approaches: 1) replacing the depth separable convolution, and 2) adding the attention mechanism module, so that the computation cost was decreased. Based on the improvement, the quantity of fruit was estimated and the total area of the bounding box of apples were respectively obtained as output. Then, these results were used as the input of the yield fitting network and actual yields were applied as the output to train the yield fitting network. The final model of fruit tree production estimation was obtained by combining the improved YOLOv5 network and the yield fitting network. Yield estimation experimental results showed that the improved YOLOv5 fruit detection algorithm could improve the recognition accuracy and the degree of lightweight. Compared with the previous algorithm, the detection speed of the algorithm proposed in this research was increased by up to 15.37%, while the mean of average accuracy (mAP) was raised up to 96.79%. The test results based on different data sets showed that the lighting conditions, coloring time and with white cloth in background had a certain impact on the accuracy of the algorithm. In addition, the yield fitting network performed better on predicting the yield of apple trees. The coefficients of determination in the training set and test set were respectively 0.7967 and 0.7982. The prediction accuracy of different yield samples was generally stable. Meanwhile, in terms of the with/without of white cloth in background, the range of relative error of the fruit tree yield measurement model was respectively within 7% and 13%. The yield estimation method of apple tree based on improved lightweight YOLOv5 had good accuracy and effectiveness, which could achieve yield estimation of apples in the natural environment, and would provide a technical reference for intelligent agricultural equipment in modern orchard environment.

Key words: apple in-situ yield estimation, deep learning, fruit detection, BP neural network, YOLOv5

CLC Number:

S252+.9

LI Zhijun, YANG Shenghui, SHI Deshuai, LIU Xingxing, ZHENG Yongjun. Yield Estimation Method of Apple Tree Based on Improved Lightweight YOLOv5[J]. Smart Agriculture, 2021, 3(2): 100-114.

Figures/Tables 21

Fig.1

Fig. 2

Fig.3

Fig. 4

Fig. 5

Table 1

Fig. 6

Fig. 7

Fig. 8

Fig. 9

Fig.10

Table 2

Table 3

Fig. 11

Table 4

Fig. 12

Table 5

Table 6

The performances of BP yield measurement module

参数	训练集	测试集
$R$	0.8979	0.8864
$R 2$	0.7967	0.7982
$R M S E$ /kg	1.5317	1.4021
$M A E$ /kg	1.1259	1.0253
$M A P E$ /%	6.3372	6.2524

Table 6

Fig. 13

Table 7

Relative error of yield measurement model with white cloth in background

评价指标	第一组	第二组	第三组	第四组	第五组	第六组	第七组	第八组	第九组	第十组
相对误差 $δ$ /％	3.05	4.25	-4.17	3.75	-5.81	4.10	-6.13	-5.39	5.41	4.28

Table 7

Table 8

Relative error of yield measurement model without white cloth in background

评价指标	第一组	第二组	第三组	第四组	第五组	第六组	第七组	第八组	第九组	第十组
相对误差 $δ$ /％	-10.34	9.72	12.15	9.83	11.13	-10.54	-12.71	8.28	11.67	-9.37

Table 8

References 25

1	黑龙江省佳木斯农业学校, 江苏省苏州农业学校.果树栽培学总论[M]. 北京: 中国农业出版社, 2009.
	Jiamusi Agricultural School of Heilongjiang province, Suzhou Agricultural School of Jiangsu province. General introduction to fruit cultivation[M]. Beijing: China Agriculture Press, 2009.
2	王少敏, 张毅, 高华军, 等. 苹果套袋栽培技术[M]. 济南: 山东科学技术出版社, 2006.
	WANG S, ZHANG Y, GAO H, et al. Apple bagging cultivation technology[M]. Jinan: Shandong Science and Technology Press, 2006.
3	PAPAGEORGIOU E I, AGGELOPOULOU K D, GEMTOS T A, et al. Yield prediction in apples using fuzzy cognitive map learning approach[J]. Computers and Electronics in Agriculture, 2013, 91: 19-29.
4	AGGELOPOULOU A D, BOCHTIS D, FOUNTAS S, et al. Yield prediction in apple orchards based on image processing[J]. Precision Agriculture, 2011, 12(3): 448-456.
5	ZHOU R, DAMEROW L, SUN Y, et al. Using colour features of cv.'Gala' apple fruits in an orchard in image processing to predict yield[J]. Precision Agriculture, 2012, 13(5): 568-580.
6	程洪, DAMEROW L, BLANKE M, 等. 基于树冠图像特征的苹果园神经网络估产模型[J]. 农业机械学报, 2015, 46(1): 14-19.
	CHENG H, DAMEROW L, BLANKE M, et al. Ann model for apple yield estimation based on feature of tree image[J]. Transactions of the CSAM, 2015, 46(1): 14-19.
7	CRTOMIR R, CVELBAR U, TOJNKO S, et al. Application of neural networks and image visualization for early predicted of apple yield[J]. Erwerbs-Obstbau, 2012, 54(2): 69-76.
8	ROY P, KISLAY A, PLONSKI P, et al. Vision-based preharvest yield mapping for apple orchards[J]. Computers and Electronics in Agriculture, 2019, 164: ID 104897.
9	REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: Unified, real-time objectdetection[C]// The IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, New York, USA: IEEE, 2016: 779-788.
10	REDMON J, FARHADI A. YOLO9000: Better, faster, stronger[C]// The IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, New York, USA: IEEE, 2017: 7263-7271.
11	REDMON J, FARHADI A. YOLO v3: An incremental improvement[EB/OL]. 2018. arXiv: .
12	BOCHKOVSKIY A, WANG C, LIAO H. YOLOv4: Optimal speed and accuracy of object detection[EB/OL]. 2020. arXiv: .
13	LIU W, ANGUELOV D, ERHAN D, et al. SSD: Single shot multibox detector[C]// European Conference on Computer Vision. Cham, Switzerland: Springer, 2016: 21-37.
14	ZHANG S, WEN L, BIAN X, et al. Single-shot refinement neural network for object detection[C]// The IEEE Conference on Computer Vision and Pattern Reco-gnition. Piscataway, New York, USA: IEEE, 2018: 4203-4212.
15	WANG D, ZHANG B, CAO Y, et al. SFSSD: Shallow feature fusion single shot multibox detector[C]// International Conference in Communications, Signal Processing, and Systems. Cham, Switzerland: Springer, 2019: 2590-2598.
16	GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]// The IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, New York, USA: IEEE, 2014: 580-587.
17	GIRSHICK R. Fast R-CNN[C]// Proceedings of the IEEE International Conference on Computer Vision. Piscataway, New York, USA: IEEE, 2015: 1440-1448.
18	REN S, HE K, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 39(6): 1137-1149.
19	周伟鸿, 朱思霖. 基于深度学习技术的智慧考场方案的应用探究[J]. 信息技术与信息化, 2020(12): 224-227.
	ZHOU W, ZHU S. Research on the application of smart examination room solutions based on deep learning technology[J]. Information Technology and Informatization, 2020(12): 224-227.
20	王沣. 改进YOLOv5的口罩和安全帽佩戴人工智能检测识别算法[J]. 建筑与预算, 2020(11): 67-69.
	WANG F. Artificial intelligence detection and recognition algorithm for masks and helmets based on improved YOLOv5[J]. Construction and Budget, 2020(11): 67-69.
21	WANG C Y, LIAO H Y M, YEH I H, et al. CSPNet: A new backbone that can enhance learning capability of CNN[C]// The IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Seattle, WA, USA: CVPRW, 2020: 390-391.
22	LIN T Y, DOLLAR P, GIRSHICK R, et al. Feature pyramid networks for object detect-ion[C]// The IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, New York, USA: IEEE, 2017: 2117-2125.
23	LIU S, QI L, QIN H, et al. Path aggregation network for instance segmentation[C]// The IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, New York, USA: IEEE, 2018: 8759-8768.
24	李德鑫, 闫志刚, 孙久运. 基于无人机视觉的河道漂浮垃圾分类检测技术研究[J/OL]. 金属矿山: 1-11.[2021-06-20]. .
	LI D, YAN Z, SUN J. Study on classification and detection technology of river floating garbage based on UAV vision[J/OL]. Metal Mine: 1-11. [2021-06-20]. .
25	SIFRE L, MALLAT S. Rigid-motion scattering for texture classification[J]. Computer Science, 2014, 3559: 501-515.

着色天数/d	训练集		测试集1						测试集2
	训练集		顺光子集		侧光子集		逆光子集		测试集2
	图片数量/张	目标框数量/个	图片数量/张	顺光目标框数量/个	图片数量/张	侧光目标框数量/个	图片数量/张	逆光目标框数量/个	图片数量/张	目标框数量/个
1	600	15，017	100	3282	100	3027	100	2963	100	3125
8	600	16，639	100	3155	100	3241	100	2834	100	3272
15	600	15，892	100	3268	100	3114	100	3052	100	3136
总计	1800	47，548	300	9705	300	9382	300	8849	300	9533

算法		检测速度及相对提升率	YOLOv5s	YOLOv5m	YOLOv5l	YOLOv5x
改进前		检测速度/ms	37.16 0.00	84.87 0.00	152.33 0.00	310.53 0.00
改进前		相对提升率/%	37.16 0.00	84.87 0.00	152.33 0.00	310.53 0.00
改进后	单独更换深度可分离卷积	检测速度/ms	30.73 -17.28	70.85 -16.51	132.26 13.17	275.54 -11.26
	单独更换深度可分离卷积	相对提升率/%	30.73 -17.28	70.85 -16.51	132.26 13.17	275.54 -11.26
	单独嵌入注意力机制模块	检测速度/ms	37.83 1.80	87.24 2.79	155.87 2.32	318.66 2.61
	单独嵌入注意力机制模块	相对提升率/%	37.83 1.80	87.24 2.79	155.87 2.32	318.66 2.61
	融合深度可分离卷积和注意力机制模块	检测速度/ms	31.44 -15.37	72.90 -14.10	136.28 -10.50	281.49 -9.35
	融合深度可分离卷积和注意力机制模块	相对提升率/%	31.44 -15.37	72.90 -14.10	136.28 -10.50	281.49 -9.35

算法		平均准确率及绝对提升率	YOLOv5s	YOLOv5m	YOLOv5l	YOLOv5x
改进前		平均准确率	89.83 0.00	90.75 0.00	92.07 0.00	93.44 0.00
改进前		绝对提升率	89.83 0.00	90.75 0.00	92.07 0.00	93.44 0.00
改进后	单独更换深度可分离卷积	平均准确率	88.94 -0.89	90.05 -0.70	91.55 -0.52	92.77 -0.67
	单独更换深度可分离卷积	绝对提升率	88.94 -0.89	90.05 -0.70	91.55 -0.52	92.77 -0.67
	单独嵌入注意力机制模块	平均准确率	92.10 2.27	94.62 3.87	95.63 3.56	96.45 3.01
	单独嵌入注意力机制模块	绝对提升率	92.10 2.27	94.62 3.87	95.63 3.56	96.45 3.01
	融合深度可分离卷积和注意力机制模块	平均准确率	92.88 3.05	93.99 3.24	95.15 3.08	96.27 2.83
	融合深度可分离卷积和注意力机制模块	绝对提升率	92.88 3.05	93.99 3.24	95.15 3.08	96.27 2.83

模型	着色1 d数据集			着色8 d数据集			着色15 d数据集
模型	顺光	侧光	逆光	顺光	侧光	逆光	顺光	侧光	逆光
改进型YOLOv5	90.25	92.48	89.31	93.56	95.08	92.95	95.26	96.79	94.07
YOLOv5	89.56	90.57	87.35	91.78	92.23	90.57	93.57	94.29	92.38
YOLOv3	87.82	88.11	86.77	89.93	90.86	88.75	92.27	93.33	91.43
SSD	86.26	86.98	83.67	87.69	89.54	88.37	90.25	91.33	89.40

数据集	YOLOv5s	YOLOv5m	YOLOv5l	YOLOv5x
着色1 d	81.47	83.26	84.63	85.89
着色8 d	83.54	86.72	87.39	88.17
着色15 d	84.76	87.93	89.56	90.72