欢迎您访问《智慧农业(中英文)》官方网站! English

Smart Agriculture ›› 2023, Vol. 5 ›› Issue (2): 35-44.doi: 10.12133/j.smartag.SA202211007

• 专题--机器视觉与农业智能感知 • 上一篇    下一篇

基于轻量级MobileNet V3-YOLOv4的生长期菠萝成熟度分析

李阳德1,2(), 马晓慧3(), 王骥3,4()   

  1. 1.广东省电信规划设计院有限公司第四分公司,广东 湛江 524088
    2.华中科技大学 电子信息与通信学院,湖北 武汉 430070
    3.广东海洋大学电子与信息工程学院,广东 湛江 524088
    4.广东省智慧海洋传感网及其装备工程技术研究中心,广东 湛江 524088
  • 收稿日期:2022-11-17 出版日期:2023-06-30
  • 基金项目:
    广东省人工智能领域专项(2019KZDZX1046)
  • 作者简介:李阳德,高级工程师,研究方向为物联网应用及智慧城市建设研究。E-mail:liyangde@gpdi.com
  • 通信作者: 马晓慧,硕士,研究方向为农业人工智能。E-mail:2068813294@qq.com
    王 骥,硕士,教授,研究方向为无线传感器网络、海洋物联网与人工智能等;E-mail:13902576499@163.com

Pineapple Maturity Analysis in Natural Environment Based on MobileNet V3-YOLOv4

LI Yangde1,2(), MA Xiaohui3(), WANG Ji3,4()   

  1. 1.The Fourth Branch of Guangdong Telecom Planning and Design Institute Co. , LTD, Zhanjiang 524088, China
    2.Huazhong University of Science and Technology, Wuhan 430070, China
    3.College of Electronic and Information Engineering, Guangdong Ocean University, Zhanjiang 524088, China
    4.Guangdong Intelligent Ocean Sensor Network and Equipment Engineering Technology Research Center, Zhanjiang 524088, China
  • Received:2022-11-17 Online:2023-06-30
  • Foundation items:Special Project in the Field of Artificial Intelligence in Guangdong Province (2019KZDZX1046)
  • About author:LI Yangde, E-mail:liyangde@gpdi.com
  • Corresponding author:

    1. MA Xiaohui, E-mail:2068813294@qq.com

    2. WANG Ji, E-mail:13902576499@163.com

摘要:

[目的/意义] 菠萝的贮藏性与成熟度相关,菠萝采摘前对其成熟度进行识别尤为重要。本研究目的在于提出一种新型网络模型,提高菠萝成熟度自动识别的准确率和速度。 [方法] 首先针对菠萝训练数据集样本少与实时性差等不足,利用在自然环境下拍摄的菠萝照片,自建了种植区场景菠萝成熟度分析数据集。之后将YOLOv4骨干网络替换成轻量级网络MobileNet V3,提出了轻量级的MobileNet V3-YOLOv4网络。同时训练了原YOLOv4模型、MobileNet V1-YOLOv4模型、MobileNet V2-YOLOv4模型以及Faster R-CNN、YOLOv3、SSD300、Retinanet、Centernet等五种不同的单、双阶段网络模型,并对比模型的评价指标,分析本文模型的优越性。[结果和讨论]试验结果表明,MobileNet V3-YOLOv4训练时间为11,924 s,参数量为53.7 MB,训练好的MobileNet V3-YOLOv4在验证集的平均精度均值(mean Average Precision,mAP)为90.92%,对于黄熟期菠萝和青熟期菠萝两种类别的检测精确率(Precision)分别为100%和98.85%,平均精度(Average Precision,AP)值分别为87.62%、94.21%,召回率(Recall)分别为77.55%、86.00%,F1分数(F1 Score)分别为0.87和0.92,推理速度(Frames Per Second,FPS)80.85 img/s。 [结论] 本研究提出的MobileNet V3-YOLOv4实现了在降低训练速度、减小参数量的同时,提高了菠萝成熟度识别的精度和推理速度,满足实际检测需求。

关键词: 菠萝成熟度, 骨干网络, MobileNet V3-YOLOv4, Faster R-CNN, SSD300, Retinanet, Centernet, 轻量级

Abstract:

[Objective] Pineapple is a common tropical fruit, and its ripeness has an important impact on the storage and marketing. It is particularly important to analyze the maturity of pineapple fruit before picking. Deep learning technology can be an effective method to achieve automatic recognition of pineapple maturity. To improve the accuracy and rate of automatic recognition of pineapple maturity, a new network model named MobileNet V3-YOLOv4 was proposed in this study. [Methods] Firstly, pineapple maturity analysis data set was constructed. A total of 1580 images were obtained, with 1264 images selected as the training set, 158 images as the validation set, and 158 images as the test set. Pineapple photos were taken in natural environment. In order to ensure the diversity of the data set and improve the robustness and generalization of the network, pineapple photos were taken under the influence of different factors such as branches and leaves occlusion, uneven lighting, overlapping shadows, etc. and the location, weather and growing environment of the collection were different. Then, according to the maturity index of pineapple, the photos of pineapple with different maturity were marked, and the labels were divided into yellow ripeness and green ripeness. The annotated images were taken as data sets and input into the network for training. Aiming at the problems of the traditional YOLOv4 network, such as large number of parameters, complex network structure and slow reasoning speed, a more optimized lightweight MobileNet V3-YOLOv4 network model was proposed. The model utilizes the benck structure to replace the Resblock in the CSPDarknet backbone network of YOLOv4. Meanwhile, in order to verify the effectiveness of the MobileNet V3-YOLOv4 network, MobileNet V1-YOLOv4 model and MobileNet V2-YOLOv4 model were also trained. Five different single-stage and two-stage network models, including R-CNN, YOLOv3, SSD300, Retinanet and Centernet were compared with each evaluation index to analyze the performance superiority of MobileNet V3-YOLOv4 model. Results and Discussions] MobileNet V3-YOLOv4 was validated for its effectiveness in pineapple maturity detection through experiments comparing model performance, model classification prediction, and accuracy tests in complex pineapple detection environments.The experimental results show that, in terms of model performance comparison, the training time of MobileNet V3-YOLOv4 was 11,924 s, with an average training time of 39.75 s per round, the number of parameters was 53.7 MB, resulting in a 25.59% reduction in the saturation time compared to YOLOv4, and the parameter count accounted for only 22%. The mean average precision (mAP) of the trained MobileNet V3-YOLOv4 in the verification set was 53.7 MB. In order to validate the classification prediction performance of the MobileNet V3-YOLOv4 model, four metrics, including Recall score, F1 Score, Precision, and average precision (AP), were utilized to classify and recognize pineapples of different maturities. The experimental results demonstrate that MobileNet V3-YOLOv4 exhibited significantly higher Precision, AP, and F1 Score the other. For the semi-ripe stage, there was a 4.49% increase in AP, 0.07 improvement in F1 Score, 1% increase in Recall, and 3.34% increase in Precision than YOLOv4. As for the ripe stage, there was a 6.06% increase in AP, 0.13 improvement in F1 Score, 16.55% increase in Recall, and 6.25% increase in Precision. Due to the distinct color features of ripe pineapples and their easy differentiation from the background, the improved network achieved a precision rate of 100.00%. Additionally, the mAP and reasoning speed (Frames Per Second, FPS) of nine algorithms were examined. The results showed that MobileNet V3-YOLOv4 achieved an mAP of 90.92%, which was 5.28% higher than YOLOv4 and 3.67% higher than YOLOv3. The FPS was measured at 80.85 img/s, which was 40.28 img/s higher than YOLOv4 and 8.91 img/s higher than SSD300. The detection results of MobileNet V3-YOLOv4 for pineapples of different maturities in complex environments indicated a 100% success rate for both the semi-ripe and ripe stages, while YOLOv4, MobileNet V1-YOLOv4, and MobileNet V2-YOLOv4 exhibited varying degrees of missed detections. [Conclusions] Based on the above experimental results, it can be concluded that MobileNet V3-YOLOv4 proposed in this study could not only reduce the training speed and parameter number number, but also improve the accuracy and reasoning speed of pineapple maturity recognition, so it has important application prospects in the field of smart orchard. At the same time, the pineapple photo data set collected in this research can also provide valuable data resources for the research and application of related fields.

Key words: pineapple maturity, backbone network, MobileNet V3-YOLOv4, Faster R-CNN, SSD300, Retinanet, Centernet, lightweight

中图分类号: