Detection Method for Dragon Fruit in Natural Environment Based on Improved YOLOX

doi:10.12133/j.smartag.SA202207001

Abstract

Abstract:

Dragon fruit detection in natural environment is the prerequisite for fruit harvesting robots to perform harvesting. In order to improve the harvesting efficiency, by improving YOLOX (You Only Look Once X) network, a target detection network with an attention module was proposed in this research. As the benchmark, YOLOX-Nano network was chose to facilitate deployment on embedded devices, and the convolutional block attention module (CBAM) was added to the backbone feature extraction network of YOLOX-Nano, which improved the robustness of the model to dragon fruit target detection to a certain extent. The correlation of features between different channels was learned by weight allocation coefficients of features of different scales, which were extracted for the backbone network. Moreover, the transmission of deep information of network structure was strengthened, which aimed at reducing the interference of dragon fruit recognition in the natural environment as well as improving the accuracy and speed of detection significantly. The performance evaluation and comparison test of the method were carried out. The results showed that, after training, the dragon fruit target detection network got an AP_0.5 value of 98.9% in the test set, an AP_0.5:0.95 value of 72.4% and F₁ score was 0.99. Compared with other YOLO network models under the same experimental conditions, on the one hand, the improved YOLOX-Nano network model proposed in this research was more lightweight, on the other hand, the detection accuracy of this method surpassed that of YOLOv3, YOLOv4 and YOLOv5 respectively. The average detection accuracy of the improved YOLOX-Nano target detection network was the highest, reaching 98.9%, 26.2% higher than YOLOv3, 9.8% points higher than YOLOv4-Tiny, and 7.9% points higher than YOLOv5-S. Finally, real-time tests were performed on videos with different input resolutions. The improved YOLOX-Nano target detection network proposed in this research had an average detection time of 21.72 ms for a single image. In terms of the size of the network model was only 3.76 MB, which was convenient for deployment on embedded devices. In conclusion, not only did the improved YOLOX-Nano target detection network model accurately detect dragon fruit under different lighting and occlusion conditions, but the detection speed and detection accuracy showed in this research could able to meet the requirements of dragon fruit harvesting in natural environment requirements at the same time, which could provide some guidance for the design of the dragon fruit harvesting robot.

Key words: fruits picking, natural environment, dragon fruit, object detection, YOLOX, attention mechanism, deep learning

CLC Number:

TP18

SHANG Fengnan, ZHOU Xuecheng, LIANG Yingkai, XIAO Mingwei, CHEN Qiao, LUO Chendi. Detection Method for Dragon Fruit in Natural Environment Based on Improved YOLOX[J]. Smart Agriculture, 2022, 4(3): 120-131.

Figures/Tables 15

Fig. 1

Fig. 2

Fig. 3

Table 1

Fig. 4

Table 2

Comparison of YOLOX test results of different model sizes

模型	大小/MB	F $1 0.5$	AP_0.5：0.95/%	AP_0.5/%
YOLOX-Nano	3.7	0.98	70.2	98.0
YOLOX-Tiny	19.4	0.97	66.3	95.7
YOLOX-S	34.3	0.97	67.6	96.4

Table 2

Fig. 5

Fig. 6

Table 3

Comparison of dragon fruit detection results of improved YOLOX-Nano network

改进的网络模型	模型大小/MB	平均时间/ms	F $1 0.5$	AP_0.5/%	AP_0.5：0.95/%	AP_S/%	AP_M/%	AP_L/%
YOLOX-Nano	3.70	18.46	0.98	98.0	70.2	55.4	60.8	76.2
YOLOX-Nano+CBAM	3.76	21.72	0.99	98.9	72.4	56.2	63.3	78.0

Table 3

Table 4

Comparison of dragon fruit detection results of different networks

网络模型	模型大小/MB	帧率/（f·s^-1）	平均时间/ms	F $1 0.5$	AP_0.5/%	AP_0.5：0.95/%	AP_S/%	AP_M/%	AP_L/%
YOLOX-Nano+CBAM	3.76	46	21.72	0.99	98.9	72.4	56.2	63.3	78.0
YOLOv5-S	27.10	59	16.87	0.93	91.0	59.5	29.0	49.8	66.5
MobileNetV3-YOLOv4	53.70	44	22.66	0.94	91.8	55.4	38.8	44.8	62.6
YOLOv4-Tiny	22.40	145	6.88	0.91	89.1	54.4	25.5	42.6	62.2
YOLOv3	235.00	51	19.38	0.83	72.7	41.4	1.7	31.0	49.7

Table 4

Fig. 7

Fig. 8

Fig. 9

Table 5

Fig. 10

References 25

1	曾曦, 胡桂兵, 秦永华. 广东省火龙果产业发展现状及对策[J]. 中国果业信息, 2019, 36(9): 9-12.
2	徐磊磊, 金琰, 侯媛媛, 等. 我国火龙果市场与产业调查分析报告[J]. 农产品市场, 2021(8): 43-45.
3	陆华忠, 李君, 李灿. 果园机械化生产技术研究进展[J]. 广东农业科学, 2020, 47(11): 226-235.
	LU H, LI J, LI C. Research progress in orchard mechanization production technology[J]. Guangdong Agricultural Sciences, 2020, 47(11): 226-235.
4	郑太雄, 江明哲, 冯明驰. 基于视觉的采摘机器人目标识别与定位方法研究综述[J]. 仪器仪表学报, 2021, 42(9): 28-51.
	ZHENG T, JIANG M, FENG M. Vision based target recognition and location for picking robot: A review[J].Chinese Journal of Scientific Instrument, 2021, 42(9): 28-51.
5	G-M JORDI, S-C RICARDO, R-P JOAN R, et al. Fruit detection and 3D location using instance segmentation neural networks and structure-from-motion photogrammetry[J]. Computers and Electronics in Agriculture, 2020, 169: ID 105165.
6	李志军, 杨圣慧, 史德帅, 等. 基于轻量化改进YOLOv5的苹果树产量测定方法[J]. 智慧农业(中英文), 2021, 3(2): 100-114.
	LI Z, YANG S, SHI D, et al. Yield estimation method of apple tree based on improved lightweight YOLOv5[J]. Smart Agriculture, 2021, 3(2): 100-114.
7	JI J, XUZ, MA H, et al. Apple fruit recognition based on a deep learning algorithm using an improved lightweight network[J]. Applied Engineering in Agriculture, 2021, 37(1): 123-134.
8	穆龙涛, 高宗斌, 崔永杰, 等. 基于改进AlexNet的广域复杂环境下遮挡猕猴桃目标识别[J]. 农业机械学报, 2019, 50(10): 24-34.
	MU L, GAO Z, CUI Y, et al. Kiwifruit detection of far-view and occluded fruit based on improved AlexNet[J].Transactions of the CSAM, 2019, 50(10): 24-34.
9	陈燕, 王佳盛, 曾泽钦, 等. 大视场下荔枝采摘机器人的视觉预定位方法[J]. 农业工程学报, 2019, 35(23): 48-54.
	CHEN Y, WANG J, ZENG Z, et al. Vision pre-positioning method for litchi picking robot under large field of view[J].Transactions of the CSAE, 2019, 35(23): 48-54.
10	彭红星, 李荆, 徐慧明, 等. 基于多重特征增强与特征融合SSD的荔枝检测[J]. 农业工程学报, 2022, 38(4): 169-177.
	PENG H, LI J, XU H, et al. Litchi detection based on multiple feature enhancement and feature fusion SSD[J]. Transactions of the CSAE, 2022, 38(4): 169-177.
11	ROY ARUNABHA M, BHADURI JAYABRATA. Real-time growth stage detection model for high degree of occultation using DenseNet-fused YOLOv4[J]. Computers and Electronics in Agriculture, 2022, 193: ID 106694.
12	XU Z, JIA R, SUN H, et al. Light-YOLOv3: Fast method for detecting green mangoes in complex scenes using picking robots[J]. Applied Intelligence, 2020, 50(12): 4670-4687.
13	XU P, FANG N, LIU N, et al. Visual recognition of cherry tomatoes in plant factory based on improved deep instance segmentation[J]. Computers and Electronics in Agriculture, 2022, 197: ID 106991.
14	周云成, 许童羽, 郑伟, 等. 基于深度卷积神经网络的番茄主要器官分类识别方法[J]. 农业工程学报, 2017, 3(15): 219-226.
	ZHOU Y, XU T, ZHENG W, et al. Classification and recognition approaches of tomato main organs based on DCNN[J]. Transactions of the CSAE, 2017, 33(15): 219-226.
15	ZHENG T, JIANG M, LI Y, et al. Research on tomato detection in natural environment based on RC-YOLOv4[J]. Computers and Electronics in Agriculture, 2022, 198: ID 107029.
16	赵春江, 文朝武, 林森, 等. 基于级联卷积神经网络的番茄花期识别检测方法[J]. 农业工程学报, 2020, 36(24): 143-152.
	ZHAO C, WEN C, LIN S, et al. Tomato florescence recognition and detection method based on cascaded neural network[J]. Transactions of the CSAE, 2020, 36(24): 143-152.
17	LI X, QIN Y, WANG F, et al. Pitaya detection in orchards using the MobileNet-YOLO model, 2020[C]// Technical Committee on Control Theory. Beijing, China: Chinese Association of Automation, 2020.
18	王金鹏, 高凯, 姜洪喆, 等. 基于改进的轻量化卷积神经网络火龙果检测方法(英文)[J]. 农业工程学报, 2020, 36(20): 218-225.
	WANG J, GAO K, JIANG H, et al. Method for detecting dragon fruit based on improved lightweight convolutional neural network[J]. Transactions of the CSAE,2020, 36(20): 218-225.
19	GE Z, LIU S, WANG F, et al. YOLOX: Exceeding YOLO series in 2021[J/OL]. arXiv: , 2021.
20	ZHANG J, KE S. Improved YOLOX fire scenario detection method[J]. Wireless Communications and Mobile Computing, 2022, 2022: 1-8.
21	LIU B, HUANG J, LIN S, et al. Improved YOLOX-S abnormal condition detection for power transmission line corridors, 2021[C]// 2021 IEEE 3rd International Conference on Power Data Science (ICPDS).Piscataway, New York, USA: IEEE, 2021: 13-16..
22	LIU M, ZHU C. Residual YOLOX-based Ship Object Detection Method, 2022[C]// 2022 2nd International Conference on Consumer Electronics and Computer Engineering (ICCECE).Piscataway, New York, USA: IEEE, 2022: 427-431.
23	WANG J, TANG C, LI J. Towards real-time analysis of marine phytoplankton images sampled at high frame rate by a YOLOX- based object detection algorithm[C]// OCEANS 2022-Chennai.Piscataway, New York, USA: IEEE, 2022: 1-9.
24	WOO S, PARK L, LEE J Y, et al. CBAM: convolutional block attention module[J/OL]. arXiv: 1807. 06521, 2018.
25	FU L, WU F, ZOU X, et al. Fast detection of banana bunches and stalks in the natural environment based on deep learning[J]. Computers and Electronics in Agriculture, 2022, 194: ID 106800.

数据集	图片总数量/张	目标个数/个
训练集	4237	11,865
验证集	471	1340
测试集	524	1396
总数	5232	14,601

分辨率	相机帧率/（f·s^-1）	平均检测帧率/（f·s^-1）
4416×1242	15	5.77
3840×1080	30	6.89
2560×720	60	12.34
1340×376	100	20.94

[1]	ZHANG Zhibo, ZHAO Xining, GAO Xiaodong, ZHANG Li, YANG Menghao. Accurate Extraction of Apple Orchard on the Loess Plateau Based on Improved Linknet Network [J]. Smart Agriculture, 2022, 4(3): 95-107.
[2]	HE Ruimin, ZHENG Kefeng, WEI Qinyang, ZHANG Xiaobin, ZHANG Jun, ZHU Yihang, ZHAO Yiying, GU Qing. Identification and Counting of Silkworms in Factory Farm Using Improved Mask R-CNN Model [J]. Smart Agriculture, 2022, 4(2): 163-173.
[3]	ZHUANG Jiayu, XU Shiwei, LI Yang, XIONG Lu, LIU Kebao, ZHONG Zhiping. Supply and Demand Forecasting Model of Multi-Agricultural Products Based on Deep Learning [J]. Smart Agriculture, 2022, 4(2): 174-182.
[4]	SHAO Mingyue, ZHANG Jianhua, FENG Quan, CHAI Xiujuan, ZHANG Ning, ZHANG Wenrong. Research Progress of Deep Learning in Detection and Recognition of Plant Leaf Diseases [J]. Smart Agriculture, 2022, 4(1): 29-46.
[5]	GUO Xiuming, ZHU Yeping, LI Shijuan, ZHANG Jie, LYU Chunyang, LIU Shengping. Scale Adaptive Small Objects Detection Method in Complex Agricultural Environment: Taking Bees as Research Object [J]. Smart Agriculture, 2022, 4(1): 140-149.
[6]	YANG Guofeng, HE Yong, FENG Xuping, LI Xiyao, ZHANG Jinnuo, YU Zeyu. Methods and New Research Progress of Remote Sensing Monitoring of Crop Disease and Pest Stress Using Unmanned Aerial Vehicle [J]. Smart Agriculture, 2022, 4(1): 1-16.
[7]	LONG Jiehua, GUO Wenzhong, LIN Sen, WEN Chaowu, ZHANG Yu, ZHAO Chunjiang. Strawberry Growth Period Recognition Method Under Greenhouse Environment Based on Improved YOLOv4 [J]. Smart Agriculture, 2021, 3(4): 99-110.
[8]	LI Zhijun, YANG Shenghui, SHI Deshuai, LIU Xingxing, ZHENG Yongjun. Yield Estimation Method of Apple Tree Based on Improved Lightweight YOLOv5 [J]. Smart Agriculture, 2021, 3(2): 100-114.
[9]	FLORES Paulo, ZHANG Zhao. Wheat Lodging Ratio Detection Based on UAS Imagery Coupled with Different Machine Learning and Deep Learning Algorithms [J]. Smart Agriculture, 2021, 3(2): 23-34.
[10]	QIU Wenjie, YE Jin, HU Liangqing, YANG Juan, LI Qili, MO Jianyou, YI Wanmao. Distilled-MobileNet Model of Convolutional Neural Network Simplified Structure for Plant Disease Recognition [J]. Smart Agriculture, 2021, 3(1): 109-117.
[11]	LI Liangde, WANG Xiujuan, KANG Mengzhen, HUA Jing, FAN Menghan. Agricultural Named Entity Recognition Based on Semantic Aggregation and Model Distillation [J]. Smart Agriculture, 2021, 3(1): 118-128.
[12]	GUO Wei, WU Huarui, ZHU Huaji. Design and Application of Facility Greenhouse Image Collecting and Environmental Data Monitoring Robot System [J]. Smart Agriculture, 2020, 2(3): 48-60.
[13]	LI Hualong, LI Miao, ZHAN Kai, LIU Xianwang, YANG Xuanjiang, HU Zelin, GUO Panpan. Construction Method and Performance Test of Prediction Model for Laying Hen Breeding Environmental Quality Evaluation [J]. Smart Agriculture, 2020, 2(3): 37-47.
[14]	WEI Jing, WANG Yuting, YUAN Huizhu, ZHANG Menglei, WANG Zhenying. Identification and Morphological Analysis of Adult Spodoptera Frugiperda and Its Close Related Species Using Deep Learning [J]. Smart Agriculture, 2020, 2(3): 75-85.
[15]	FLORES Paulo, ZHANG Zhao, MATHEW Jithin, JAHAN Nusrat, STENGER John. Distinguishing Volunteer Corn from Soybean at Seedling Stage Using Images and Machine Learning [J]. Smart Agriculture, 2020, 2(3): 61-74.