Pineapple Maturity Analysis in Natural Environment Based on MobileNet V3-YOLOv4

doi:10.12133/j.smartag.SA202211007

Abstract

Abstract:

[Objective] Pineapple is a common tropical fruit, and its ripeness has an important impact on the storage and marketing. It is particularly important to analyze the maturity of pineapple fruit before picking. Deep learning technology can be an effective method to achieve automatic recognition of pineapple maturity. To improve the accuracy and rate of automatic recognition of pineapple maturity, a new network model named MobileNet V3-YOLOv4 was proposed in this study. [Methods] Firstly, pineapple maturity analysis data set was constructed. A total of 1580 images were obtained, with 1264 images selected as the training set, 158 images as the validation set, and 158 images as the test set. Pineapple photos were taken in natural environment. In order to ensure the diversity of the data set and improve the robustness and generalization of the network, pineapple photos were taken under the influence of different factors such as branches and leaves occlusion, uneven lighting, overlapping shadows, etc. and the location, weather and growing environment of the collection were different. Then, according to the maturity index of pineapple, the photos of pineapple with different maturity were marked, and the labels were divided into yellow ripeness and green ripeness. The annotated images were taken as data sets and input into the network for training. Aiming at the problems of the traditional YOLOv4 network, such as large number of parameters, complex network structure and slow reasoning speed, a more optimized lightweight MobileNet V3-YOLOv4 network model was proposed. The model utilizes the benck structure to replace the Resblock in the CSPDarknet backbone network of YOLOv4. Meanwhile, in order to verify the effectiveness of the MobileNet V3-YOLOv4 network, MobileNet V1-YOLOv4 model and MobileNet V2-YOLOv4 model were also trained. Five different single-stage and two-stage network models, including R-CNN, YOLOv3, SSD300, Retinanet and Centernet were compared with each evaluation index to analyze the performance superiority of MobileNet V3-YOLOv4 model. Results and Discussions] MobileNet V3-YOLOv4 was validated for its effectiveness in pineapple maturity detection through experiments comparing model performance, model classification prediction, and accuracy tests in complex pineapple detection environments.The experimental results show that, in terms of model performance comparison, the training time of MobileNet V3-YOLOv4 was 11,924 s, with an average training time of 39.75 s per round, the number of parameters was 53.7 MB, resulting in a 25.59% reduction in the saturation time compared to YOLOv4, and the parameter count accounted for only 22%. The mean average precision (mAP) of the trained MobileNet V3-YOLOv4 in the verification set was 53.7 MB. In order to validate the classification prediction performance of the MobileNet V3-YOLOv4 model, four metrics, including Recall score, F₁ Score, Precision, and average precision (AP), were utilized to classify and recognize pineapples of different maturities. The experimental results demonstrate that MobileNet V3-YOLOv4 exhibited significantly higher Precision, AP, and F₁ Score the other. For the semi-ripe stage, there was a 4.49% increase in AP, 0.07 improvement in F₁ Score, 1% increase in Recall, and 3.34% increase in Precision than YOLOv4. As for the ripe stage, there was a 6.06% increase in AP, 0.13 improvement in F₁ Score, 16.55% increase in Recall, and 6.25% increase in Precision. Due to the distinct color features of ripe pineapples and their easy differentiation from the background, the improved network achieved a precision rate of 100.00%. Additionally, the mAP and reasoning speed (Frames Per Second, FPS) of nine algorithms were examined. The results showed that MobileNet V3-YOLOv4 achieved an mAP of 90.92%, which was 5.28% higher than YOLOv4 and 3.67% higher than YOLOv3. The FPS was measured at 80.85 img/s, which was 40.28 img/s higher than YOLOv4 and 8.91 img/s higher than SSD300. The detection results of MobileNet V3-YOLOv4 for pineapples of different maturities in complex environments indicated a 100% success rate for both the semi-ripe and ripe stages, while YOLOv4, MobileNet V1-YOLOv4, and MobileNet V2-YOLOv4 exhibited varying degrees of missed detections. [Conclusions] Based on the above experimental results, it can be concluded that MobileNet V3-YOLOv4 proposed in this study could not only reduce the training speed and parameter number number, but also improve the accuracy and reasoning speed of pineapple maturity recognition, so it has important application prospects in the field of smart orchard. At the same time, the pineapple photo data set collected in this research can also provide valuable data resources for the research and application of related fields.

Key words: pineapple maturity, backbone network, MobileNet V3-YOLOv4, Faster R-CNN, SSD300, Retinanet, Centernet, lightweight

CLC Number:

TP391

LI Yangde, MA Xiaohui, WANG Ji. Pineapple Maturity Analysis in Natural Environment Based on MobileNet V3-YOLOv4[J]. Smart Agriculture, 2023, 5(2): 35-44.

Figures/Tables 11

Fig. 1

Fig.2

Fig. 3

Fig.4

Fig. 5

Fig.6

Table 1

Table 2

Table 3

Table 4

Fig. 7

References 18

1	金琰, 刘海清, 刘恩平. 我国菠萝种植区域优势研究[J]. 中国农业资源与区划, 2014, 35(3): 100-104.
	JIN Y, LIU H Q, LIU E P. Study on the regional advantages of pineapple planting in China[J]. Chinese journal of agricultural resources and regional planning, 2014, 35(3): 100-104.
2	刘传和, 刘岩, 谢盛良, 等. 不同成熟度菠萝果实香气成分分析[J]. 热带作物学报, 2009, 30(2): 234-237.
	LIU C H, LIU Y, XIE S L, et al. Analysis of aroma components of pineapple fruit at different ripening stages[J]. Chinese journal of tropical crops, 2009, 30(2): 234-237.
3	刘胜辉, 孙伟生, 陆新华, 等. 6个菠萝品种成熟果实香气成分分析[J]. 热带作物学报, 2015, 36(6): 1179-1185.
	LIU S H, SUN W S, LU X H, et al. Analysis of aromatic components in ripe fruits of 6 pineapple cultivars[J]. Chinese journal of tropical crops, 2015, 36(6): 1179-1185.
4	FORTUNA-CERVANTES J M, RAMÍREZ-TORRESMT, MARTINEZ-CARRANZA J, et al. Object detection in aerial navigation using wavelet transform and convolutional neural networks: A first approach[J]. Proceedings of the institute for system programming of the RAS, 2021, 33(2): 149-162.
5	GAI R, CHEN N, YUAN H.A detection algorithm for cherry fruits based on the improved YOLOv4 model[J].Neural computing and application,2021,35(19):13895-13906.
6	CHEN M C, CHENG Y T, LIU C Y.Implementation of a fruit quality classification application using an artificial intelligence algorithm[J].Sensors and materials: An international journal on sensor technology, 2022,34(1),151-162.
7	KUZNETSOVA A, MALEVA T, SOLOVIEV V.Using YOLOv3 algorithm with pre- and post-processing for apple detection in fruit-harvesting robot[J]. Agronomy, 2020, 10(7): 1016-1035.
8	ZHANG Y D, DONG Z C, CHEN X Q, et al. Image based fruit category classification by 13-layer deep convolutional neural network and data augmentation[J]. Multimedia tools and applications, 2019, 78(3): 3613-3632.
9	CHAIKAEW A, THANAVANICH T, DUANGTANG P, et al. Convolutional neural network for pineapple ripeness classification machine[C]// 2019 16th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON). Piscataway, NJ, USA: IEEE, 2020: 373-376.
10	CUONGN H H, TRINHTH, MEESAD P, et al. Improved YOLO object detection algorithm to detect ripe pineapple phase[J]. Journal of intelligent& fuzzy systems: Applications in engineering andtechnology, 2022, 43(1): 1365-1381.
11	张星, 高巧明, 潘栋, 等. 基于改进YOLOv3的田间复杂环境下菠萝拾捡识别研究[J]. 中国农机化学报, 2021, 42(1): 201-206.
	ZHANG X, GAO Q M, PAN D, et al. Picking recognition research of pineapple in complex field environment based on improved YOLOv3[J]. Journal of Chinese agricultural mechanization, 2021, 42(1): 201-206.
12	LIU T H, NIE X N, WU J M, et al. Pineapple (Ananas comosus) fruit detection and localization in natural environment based on binocular stereo vision and improved YOLOv3 model[J]. Precision agriculture, 2023, 24(1): 139-160.
13	茅智慧, 朱佳利, 吴鑫, 等. 基于YOLO的自动驾驶目标检测研究综述[J]. 计算机工程与应用, 2022, 58(15): 68-77.
	MAO Z H, ZHU J L, WU X, et al. Review of YOLO based target detection for autonomous driving[J]. Computer engineering and applications, 2022, 58(15): 68-77.
14	WANG X, GAO J S, HOU B J, et al. A lightweight modified YOLOX network using coordinate attention mechanism for PCB surface defect detection[J]. IEEE sensors journal, 2022, 22(21): 20910-20920.
15	XU D Q, WU Y Q. Improved YOLOv3 with DenseNet for multi-scale remote sensing target detection[J]. Sensors, 2020, 20(15): ID 4276.
16	GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]// 2014 IEEE Conferenceon Computer Visionand Pattern Recognition. Piscataway, NJ, USA: IEEE, 2014: 580-587.
17	SUN Y Y, MA S, SUN S Y, et al. Partial discharge pattern recognition of transformers based on MobileNets convolutional neural network[J]. Applied sciences, 2021, 11(15): ID 6984.
18	LI J P, ZHU K Y, WANG F, et al. Deep neural network-based real time fish detection method in the scene of marine fishing supervision[J]. Journal of intelligent& fuzzy systems: Applications in engineering and technology, 2021, 41(3): 4527-4532.

初始参数	初始参数值
训练周期（epoch）/轮	300
批处理	8
初始学习率	0.01
动量参数	0.937

序号	网络	训练时间/s	平均每轮训练时间/s	饱和周期/轮	参数量/MB
1	Faster R-CNN	31，881	106.27	180	108.0
2	YOLOv3	20，820	69.40	170	235.0
3	SSD300	15，728	52.42	100	91.1
4	Retinanet	10，128	33.76	42	138.0
5	Centernet	16，462	54.87	160	124.0
6	YOLOv4	19，172	63.91	214	244.0
7	MobileNet V1-YOLOv4	11，000	36.67	195	51.1
8	MobileNet V2-YOLOv4	12，068	40.23	227	46.5
9	MobileNet V3-YOLOv4	11，924	39.75	256	53.7

序号	网络	青熟期菠萝				黄熟期菠萝
序号	网络	AP/%	F₁Score	Recall/%	Precision/%	AP/%	F₁Score	Recall/%	Precision/%
1	Faster R-CNN	72.36	0.76	78.09	73.33	94.41	0.94	94.95	93.62
2	YOLOv3	85.19	0.78	67.45	93.65	89.31	0.84	75.87	93.22
3	SSD300	68.00	0.81	80.37	80.70	94.20	0.96	95.83	95.64
4	Retinanet	84.65	0.8	71.96	89.24	89.10	0.84	75.14	96.35
5	Centernet	73.25	0.36	21.70	98.55	90.02	0.66	49.84	99.37
6	YOLOv4	89.72	0.85	85.00	95.51	81.56	0.74	61.00	93.75
7	MobileNet V1-YOLOv4	92.53	0.92	86.00	98.85	89.14	0.82	73.47	92.31
8	MobileNet V2-YOLOv4	92.73	0.89	89.00	98.89	87.61	0.80	79.59	100.00
9	MobileNet V3-YOLOv4	94.21	0.92	86.00	98.85	87.62	0.87	77.55	100.00

序号	网络	FPS/（img·s^-1）	mAP
1	Faster R-CNN	15.22	83.39
2	Yolov3	51.33	87.25
3	SSD300	71.94	81.1
4	Retinanet	27.17	86.88
5	Centernet	68.07	81.64
6	YOLOv4	40.57	85.64
7	MobileNet V1-YOLOv4	69.97	90.83
8	MobileNet V2-YOLOv4	75.95	90.17
9	MobileNet V3-YOLOv4	80.85	90.92

[1]	ZHAO Yu, REN Yiping, PIAO Xinru, ZHENG Danyang, LI Dongming. Lightweight Intelligent Recognition of Saposhnikovia Divaricata (Turcz.) Schischk Originality Based on Improved ShuffleNet V2 [J]. Smart Agriculture, 2023, 5(2): 104-114.
[2]	XIA Xue, CHAI Xiujuan, ZHANG Ning, ZHOU Shuo, SUN Qixin, SUN Tan. A Lightweight Fruit Load Estimation Model for Edge Computing Equipment [J]. Smart Agriculture, 2023, 5(2): 1-12.
[3]	XIA Ye, LEI Xiaohui, QI Yannan, XU Tao, YUAN Quanchun, PAN Jian, JIANG Saike, LYU Xiaolan. Detection of Pear Inflorescence Based on Improved Ghost-YOLOv5s-BiFPN Algorithm [J]. Smart Agriculture, 2022, 4(3): 108-119.
[4]	ZHANG Xiaoqing, SHAO Song, GUO Xinyu, FAN Jiangchuan. High-Throughput Dynamic Monitoring Method of Field Maize Seedling [J]. Smart Agriculture, 2021, 3(2): 88-99.
[5]	Xia Xue, Sun Qixin, Shi Xiao, Chai Xiujuan. Apple detection model based on lightweight anchor-free deep convolutional neural network [J]. Smart Agriculture, 2020, 2(1): 99-110.