Welcome to Smart Agriculture 中文
Topic--Machine Vision and Agricultural Intelligent Perception

Pineapple Maturity Analysis in Natural Environment Based on MobileNet V3-YOLOv4

  • LI Yangde ,
  • MA Xiaohui ,
  • WANG Ji
Expand
  • 1.The Fourth Branch of Guangdong Telecom Planning and Design Institute Co. , LTD, Zhanjiang 524088, China
    2.Huazhong University of Science and Technology, Wuhan 430070, China
    3.College of Electronic and Information Engineering, Guangdong Ocean University, Zhanjiang 524088, China
    4.Guangdong Intelligent Ocean Sensor Network and Equipment Engineering Technology Research Center, Zhanjiang 524088, China
LI Yangde, E-mail:liyangde@gpdi.com

1. MA Xiaohui, E-mail:2068813294@qq.com

2. WANG Ji, E-mail:13902576499@163.com

Received date: 2022-11-17

  Online published: 2023-07-06

Supported by

Special Project in the Field of Artificial Intelligence in Guangdong Province (2019KZDZX1046)

Abstract

[Objective] Pineapple is a common tropical fruit, and its ripeness has an important impact on the storage and marketing. It is particularly important to analyze the maturity of pineapple fruit before picking. Deep learning technology can be an effective method to achieve automatic recognition of pineapple maturity. To improve the accuracy and rate of automatic recognition of pineapple maturity, a new network model named MobileNet V3-YOLOv4 was proposed in this study. [Methods] Firstly, pineapple maturity analysis data set was constructed. A total of 1580 images were obtained, with 1264 images selected as the training set, 158 images as the validation set, and 158 images as the test set. Pineapple photos were taken in natural environment. In order to ensure the diversity of the data set and improve the robustness and generalization of the network, pineapple photos were taken under the influence of different factors such as branches and leaves occlusion, uneven lighting, overlapping shadows, etc. and the location, weather and growing environment of the collection were different. Then, according to the maturity index of pineapple, the photos of pineapple with different maturity were marked, and the labels were divided into yellow ripeness and green ripeness. The annotated images were taken as data sets and input into the network for training. Aiming at the problems of the traditional YOLOv4 network, such as large number of parameters, complex network structure and slow reasoning speed, a more optimized lightweight MobileNet V3-YOLOv4 network model was proposed. The model utilizes the benck structure to replace the Resblock in the CSPDarknet backbone network of YOLOv4. Meanwhile, in order to verify the effectiveness of the MobileNet V3-YOLOv4 network, MobileNet V1-YOLOv4 model and MobileNet V2-YOLOv4 model were also trained. Five different single-stage and two-stage network models, including R-CNN, YOLOv3, SSD300, Retinanet and Centernet were compared with each evaluation index to analyze the performance superiority of MobileNet V3-YOLOv4 model. Results and Discussions] MobileNet V3-YOLOv4 was validated for its effectiveness in pineapple maturity detection through experiments comparing model performance, model classification prediction, and accuracy tests in complex pineapple detection environments.The experimental results show that, in terms of model performance comparison, the training time of MobileNet V3-YOLOv4 was 11,924 s, with an average training time of 39.75 s per round, the number of parameters was 53.7 MB, resulting in a 25.59% reduction in the saturation time compared to YOLOv4, and the parameter count accounted for only 22%. The mean average precision (mAP) of the trained MobileNet V3-YOLOv4 in the verification set was 53.7 MB. In order to validate the classification prediction performance of the MobileNet V3-YOLOv4 model, four metrics, including Recall score, F1 Score, Precision, and average precision (AP), were utilized to classify and recognize pineapples of different maturities. The experimental results demonstrate that MobileNet V3-YOLOv4 exhibited significantly higher Precision, AP, and F1 Score the other. For the semi-ripe stage, there was a 4.49% increase in AP, 0.07 improvement in F1 Score, 1% increase in Recall, and 3.34% increase in Precision than YOLOv4. As for the ripe stage, there was a 6.06% increase in AP, 0.13 improvement in F1 Score, 16.55% increase in Recall, and 6.25% increase in Precision. Due to the distinct color features of ripe pineapples and their easy differentiation from the background, the improved network achieved a precision rate of 100.00%. Additionally, the mAP and reasoning speed (Frames Per Second, FPS) of nine algorithms were examined. The results showed that MobileNet V3-YOLOv4 achieved an mAP of 90.92%, which was 5.28% higher than YOLOv4 and 3.67% higher than YOLOv3. The FPS was measured at 80.85 img/s, which was 40.28 img/s higher than YOLOv4 and 8.91 img/s higher than SSD300. The detection results of MobileNet V3-YOLOv4 for pineapples of different maturities in complex environments indicated a 100% success rate for both the semi-ripe and ripe stages, while YOLOv4, MobileNet V1-YOLOv4, and MobileNet V2-YOLOv4 exhibited varying degrees of missed detections. [Conclusions] Based on the above experimental results, it can be concluded that MobileNet V3-YOLOv4 proposed in this study could not only reduce the training speed and parameter number number, but also improve the accuracy and reasoning speed of pineapple maturity recognition, so it has important application prospects in the field of smart orchard. At the same time, the pineapple photo data set collected in this research can also provide valuable data resources for the research and application of related fields.

Cite this article

LI Yangde , MA Xiaohui , WANG Ji . Pineapple Maturity Analysis in Natural Environment Based on MobileNet V3-YOLOv4[J]. Smart Agriculture, 2023 , 5(2) : 35 -44 . DOI: 10.12133/j.smartag.SA202211007

References

1 金琰, 刘海清, 刘恩平. 我国菠萝种植区域优势研究[J]. 中国农业资源与区划, 2014, 35(3): 100-104.
  JIN Y, LIU H Q, LIU E P. Study on the regional advantages of pineapple planting in China[J]. Chinese journal of agricultural resources and regional planning, 2014, 35(3): 100-104.
2 刘传和, 刘岩, 谢盛良, 等. 不同成熟度菠萝果实香气成分分析[J]. 热带作物学报, 2009, 30(2): 234-237.
  LIU C H, LIU Y, XIE S L, et al. Analysis of aroma components of pineapple fruit at different ripening stages[J]. Chinese journal of tropical crops, 2009, 30(2): 234-237.
3 刘胜辉, 孙伟生, 陆新华, 等. 6个菠萝品种成熟果实香气成分分析[J]. 热带作物学报, 2015, 36(6): 1179-1185.
  LIU S H, SUN W S, LU X H, et al. Analysis of aromatic components in ripe fruits of 6 pineapple cultivars[J]. Chinese journal of tropical crops, 2015, 36(6): 1179-1185.
4 FORTUNA-CERVANTES J M, RAMíREZ-TORRESMT, MARTINEZ-CARRANZA J, et al. Object detection in aerial navigation using wavelet transform and convolutional neural networks: A first approach[J]. Proceedings of the institute for system programming of the RAS, 2021, 33(2): 149-162.
5 GAI R, CHEN N, YUAN H.A detection algorithm for cherry fruits based on the improved YOLOv4 model[J].Neural computing and application,2021,35(19):13895-13906.
6 CHEN M C, CHENG Y T, LIU C Y.Implementation of a fruit quality classification application using an artificial intelligence algorithm[J].Sensors and materials: An international journal on sensor technology, 2022,34(1),151-162.
7 KUZNETSOVA A, MALEVA T, SOLOVIEV V.Using YOLOv3 algorithm with pre- and post-processing for apple detection in fruit-harvesting robot[J]. Agronomy, 2020, 10(7): 1016-1035.
8 ZHANG Y D, DONG Z C, CHEN X Q, et al. Image based fruit category classification by 13-layer deep convolutional neural network and data augmentation[J]. Multimedia tools and applications, 2019, 78(3): 3613-3632.
9 CHAIKAEW A, THANAVANICH T, DUANGTANG P, et al. Convolutional neural network for pineapple ripeness classification machine[C]// 2019 16th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON). Piscataway, NJ, USA: IEEE, 2020: 373-376.
10 CUONGN H H, TRINHTH, MEESAD P, et al. Improved YOLO object detection algorithm to detect ripe pineapple phase[J]. Journal of intelligent& fuzzy systems: Applications in engineering andtechnology, 2022, 43(1): 1365-1381.
11 张星, 高巧明, 潘栋, 等. 基于改进YOLOv3的田间复杂环境下菠萝拾捡识别研究[J]. 中国农机化学报, 2021, 42(1): 201-206.
  ZHANG X, GAO Q M, PAN D, et al. Picking recognition research of pineapple in complex field environment based on improved YOLOv3[J]. Journal of Chinese agricultural mechanization, 2021, 42(1): 201-206.
12 LIU T H, NIE X N, WU J M, et al. Pineapple (Ananas comosus) fruit detection and localization in natural environment based on binocular stereo vision and improved YOLOv3 model[J]. Precision agriculture, 2023, 24(1): 139-160.
13 茅智慧, 朱佳利, 吴鑫, 等. 基于YOLO的自动驾驶目标检测研究综述[J]. 计算机工程与应用, 2022, 58(15): 68-77.
  MAO Z H, ZHU J L, WU X, et al. Review of YOLO based target detection for autonomous driving[J]. Computer engineering and applications, 2022, 58(15): 68-77.
14 WANG X, GAO J S, HOU B J, et al. A lightweight modified YOLOX network using coordinate attention mechanism for PCB surface defect detection[J]. IEEE sensors journal, 2022, 22(21): 20910-20920.
15 XU D Q, WU Y Q. Improved YOLOv3 with DenseNet for multi-scale remote sensing target detection[J]. Sensors, 2020, 20(15): ID 4276.
16 GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]// 2014 IEEE Conferenceon Computer Visionand Pattern Recognition. Piscataway, NJ, USA: IEEE, 2014: 580-587.
17 SUN Y Y, MA S, SUN S Y, et al. Partial discharge pattern recognition of transformers based on MobileNets convolutional neural network[J]. Applied sciences, 2021, 11(15): ID 6984.
18 LI J P, ZHU K Y, WANG F, et al. Deep neural network-based real time fish detection method in the scene of marine fishing supervision[J]. Journal of intelligent& fuzzy systems: Applications in engineering and technology, 2021, 41(3): 4527-4532.
Outlines

/