Strawberry Growth Period Recognition Method Under Greenhouse Environment Based on Improved YOLOv4

doi:10.12133/j.smartag.2021.3.4.202109-SA006

Abstract

Abstract:

Aiming at the real-time detection and classification of the growth period of crops in the current digital cultivation and regulation technology of facility agriculture, an improved YOLOv4 method for identifying the growth period of strawberries in a greenhouse environment was proposed. The attention mechanism into the Cross Stage Partial Residual (CSPRes) module of the YOLOv4 backbone network was introduced, and the target feature information of different growth periods of strawberries while reducing the interference of complex backgrounds was integrated, the detection accuracy while ensured real-time detection efficiency was improved. Took the smart facility strawberry in Yunnan province as the test object, the results showed that the detection accuracy (AP) of the YOLOv4-CBAM model during flowering, fruit expansion, green and mature period were 92.38%, 82.45%, 68.01% and 92.31%, respectively, the mean average precision (mAP) was 83.78%, the mean inetersection over union (mIoU) was 77.88%, and the detection time for a single image was 26.13 ms. Compared with the YOLOv4-SC model, mAP and mIoU were increased by 1.62% and 2.73%, respectively. Compared with the YOLOv4-SE model, mAP and mIOU increased by 4.81% and 3.46%, respectively. Compared with the YOLOv4 model, mAP and mIOU increased by 8.69% and 5.53%, respectively. As the attention mechanism was added to the improved YOLOv4 model, the amount of parameters increased, but the detection time of improved YOLOv4 models only slightly increased. At the same time, the number of fruit expansion period recognized by YOLOv4 was less than that of YOLOv4-CBAM, YOLOv4-SC and YOLOv4-SE, because the color characteristics of fruit expansion period were similar to those of leaf background, which made YOLOv4 recognition susceptible to leaf background interference, and added attention mechanism could reduce background information interference. YOLOv4-CBAM had higher confidence and number of identifications in identifying strawberry growth stages than YOLOv4-SC, YOLOv4-SE and YOLOv4 models, indicated that YOLOv4-CBAM model can extract more comprehensive and rich features and focus more on identifying targets, thereby improved detection accuracy. YOLOv4-CBAM model can meet the demand for real-time detection of strawberry growth period status.

Key words: object detection, strawberry, growth period recognition, YOLOv4, residual module, attention mechanism, loss function

CLC Number:

S126

LONG Jiehua, GUO Wenzhong, LIN Sen, WEN Chaowu, ZHANG Yu, ZHAO Chunjiang. Strawberry Growth Period Recognition Method Under Greenhouse Environment Based on Improved YOLOv4[J]. Smart Agriculture, 2021, 3(4): 99-110.

References

1	张更, 颜志明, 王全智, 等. 我国设施草莓无土栽培技术的研究进展与发展建议[J]. 江苏农业科学, 2019, 47(18): 58-61.
1	ZHANG G, YAN Z, WANG Q, et al. Research progress and development suggestions of soilless culture techniques of China's facility strawberry[J]. Jiangsu Agricultural Sciences, 2019, 47(18): 58-61.
2	林森, 郭文忠, 郑建锋, 等. 基于知识图谱和机器视觉的智慧草莓生产托管服务系统实践[J]. 农业工程技术, 2021, 41(4): 17-20.
2	LIN S, GUO W, ZHENG J, et al. Practice of smart strawberry production hosting service system based on knowledge graph and machine vision[J]. Agricultural Engineering Technology, 2021, 41(4): 17-20.
3	徐建鹏, 王杰, 徐祥, 等. 基于 RAdam 卷积神经网络的水稻生育期图像识别[J]. 农业工程学报, 2021, 37(8): 143-150.
3	XU J, WANG J, XU X, et al. Image recognition for different developmental stages of rice by RAdam deep convolutional neural networks[J]. Transactions of the CSAE, 2021, 37(8): 143-150.
4	CHEN W, LU S, LIU B, et al. Detecting citrus in orchard environment by using improved YOLOv4[J]. Scientific Programming, 2020(1): 1-13.
5	PéREZ-BORRERO I,MARíN-SANTOS D, GEGúNDE-ARIAS M E, et al. A fast and accurate deep learning method for strawberry instance segmentation[J]. Computers and Electronics in Agriculture, 2020, 178(6): 105736-105748.
6	刘芳, 刘玉坤, 林森, 等. 基于改进型YOLO的复杂环境下番茄果实快速识别方法[J]. 农业机械学报, 2020, 51(6): 229-237.
6	LIU F, LIU Y, LIN S, et al. Fast recognition method for tomatoes under complex environments based on improved YOLO[J]. Transactions of the CSAM, 2020, 51(6): 229-237.
7	LIN P, CHEN Y. Detection of strawberry flowers in outdoor field by deep neural network[C]// 2018 IEEE 3rd International Conference on Image, Vision and Computing (ICIVC). Piscataway, New York, USA: IEEE, 2018: 482-486.
8	YU Y, ZHANG K, YANG L, et al. Fruit detection for strawberry harvesting robot in non-structural environment based on Mask-RCNN[J]. Computers and Electronics in Agriculture, 2019(163): 104846-104855.
9	李志军, 杨圣慧, 史德帅, 等. 基于轻量化改进YOLOv5的苹果树产量测定方法[J]. 智慧农业, 2021, 3(2): 100-114.
9	LI Z, YANG S, SHI D, et al. Yield estimation method of apple tree based on improved lightweight YOLOv5[J]. Smart Agriculture, 2021, 3(2): 100-114.
10	刘小刚, 范诚, 李加念, 等. 基于卷积神经网络的草莓识别方法[J]. 农业机械学报, 2020, 51(2): 237-244.
10	LIU X, FANG C, LI J, et al. Identification method of strawberry based on convolutional neural network[J]. Transactions of the CSAM, 2020, 51(2): 237-244.
11	赵春江, 文朝武, 林森, 等. 基于级联卷积神经网络的番茄花期识别检测方法[J]. 农业工程学报, 2020, 36(24): 143-152.
11	ZHAO C, WEN C, LIN S, et al. Tomato florescence recognition and detection method based on cascaded neural network[J]. Transactions of the CSAE, 2020, 36(24): 143-152.
12	刘天真, 滕桂法, 苑迎春, 等. 基于改进YOLOv3的自然场景下冬枣果实识别方法[J]. 农业机械学报, 2021, 52(5): 17-25.
12	LIU T, TENG G, YUAN Y, et al. Winter jujube fruit recognition method based on improved YOLOv3 under natural scene[J]. Transactions of the CSAM, 2021, 52(5): 17-25.
13	刘立波, 程晓龙, 赖军臣. 基于改进全卷积网络的棉田冠层图像分割方法[J]. 农业工程学报, 2018, 34(12): 193-201.
13	LIU L, CHENG X, LAI J, et al. Segmentation method for cotton canopy image based on improved fully convolutional network model[J]. Transactions of the CSAE, 2018, 34(12): 193-201.
14	朱逢乐, 郑增威. 基于图像和卷积神经网络的蝴蝶兰种苗生长势评估[J]. 农业工程学报, 2020, 36(9): 185-194.
14	ZHU F, ZHENG Z. Image-based assessment of growth vigor for Phalaenopsis aphrodite seedlings using convolutional neural network[J]. Transactions of the CSAE, 2020, 36(9): 185-194.
15	CHAUDHARI S, MITHAL V, POLATKAN G, et al. An attentive survey of attention models[J/OL]. arXiv: 1904.02874v
15	3[cs.Lg]. 2021.
16	徐诚极, 王晓峰, 杨亚东. Attention-YOLO: 引入注意力机制的YOLO检测算法[J]. 计算机工程与应用, 2019, 55(6): 13-23.
16	XU C, WANG X, YANG Y. Attention-YOLO: YOLO detection algorithm that introduces attention mechanism[J]. Computer Engineering and Applications, 2019, 55(6): 13-23.
17	HU J, SHEN L, SUN G, et al. Squeeze-and-excitation networks[C]// The IEEE conference on computer vision and pattern recognition. Piscataway, New York, USA: IEEE, 2018: 2011-2023.
18	汶茂宁. 基于轮廓波CNN和选择性注意机制的高分辨SAR目标检测和分类[D]. 西安: 西安电子科技大学, 2018.
18	WEN M. Target detection and classification for high-resolution SAR image based on contourlet CNN and selective attention mechanism[D]. Xi'an: Xidian University, 2018.
19	WOO S, PARK J, LEE J-Y, et al. CBAM: Convolutional block attention module[C]// The European Conference on Computer Vision(ECCV). Berlin, German: Springer, 2018: 3-19.
20	BOCHKOVSKIY A, WANG C, LIAO H M. YOLOv4: Optimal speed and accuracy of object detection[J/OL]. arXiv:2004.
20	10934 [cs.CV]. 2020.
21	REDMON J, FARHAD A. YOLOv3: An incremental improvement[J/OL]. arXiv:1804.
21	02767 [cs.CV]. 2018.
22	蒋镕圻, 彭月平, 谢文宣, 等. 嵌入scSE模块的改进YOLOv4小目标检测算法[J]. 图学学报, 2021, 42(4): 546-555.
22	JIANG R, PENG Y, XIE W, et al. Improved YOLOv4 small target detection algorithm with embedded scSE module[J]. Journal of Graphics, 2021, 42(4): 546-555.
23	李文涛, 张岩, 莫锦秋, 等. 基于改进YOLOv3-tiny的田间行人与农机障碍物检测[J]. 农业机械学报, 2020, 51(S1):8-15, 40.
23	LI W, ZHANG Y, MO J, et al. Detection of pedestrian and agricultural vehicles in field based on improved YOLOv3 tiny[J]. Transactions of the CSAM, 2020, 51(S1): 8-15, 40.
24	WANG C, LIAO H M, WU Y, et al. CSPNet: A new backbone that can enhance learning capability of CNN[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Piscataway, New York, USA: IEEE, 2020: 1571-1580.
25	温长吉, 娄月, 张笑然, 等. 基于改进稠密胶囊网络模型的植物识别方法[J]. 农业工程学报, 2020, 36(8): 143-155.
25	WEN C, LOU Y, ZHANG X, et al. Plant recognition method based on an improved dense CapsNet[J]. Transactions of the CSAE, 2020, 36(8): 143-155.
26	HE K, ZHANG X, REN S, et al. Deep residual Learning for image recognition[C]// The IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, New York, USA: IEEE, 2016: 770-778.
27	HE K, ZHANG X, REN S, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 37(9): 1904-1916.
28	LIU S, QI L, QIN H, et al. Path aggregation network for instance segmentation[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, New York, USA: IEEE, 2018: 8759-8768.

[1]	ZHAO Yu, REN Yiping, PIAO Xinru, ZHENG Danyang, LI Dongming. Lightweight Intelligent Recognition of Saposhnikovia Divaricata (Turcz.) Schischk Originality Based on Improved ShuffleNet V2 [J]. Smart Agriculture, 2023, 5(2): 104-114.
[2]	PAN Chenlu, ZHANG Zhenghua, GUI Wenhao, MA Jiajun, YAN Chenxi, ZHANG Xiaomin. Rice Disease and Pest Recognition Method Integrating ECA Mechanism and DenseNet201 [J]. Smart Agriculture, 2023, 5(2): 45-55.
[3]	LI Yangde, MA Xiaohui, WANG Ji. Pineapple Maturity Analysis in Natural Environment Based on MobileNet V3-YOLOv4 [J]. Smart Agriculture, 2023, 5(2): 35-44.
[4]	ZHU Haipeng, ZHANG Yu'an, LI Huanhuan, WANG Jianwen, YANG Yingkui, SONG Rende. Classification and Recognition Method for Yak Meat Parts Based on Improved Residual Network Model [J]. Smart Agriculture, 2023, 5(2): 115-125.
[5]	SHANG Fengnan, ZHOU Xuecheng, LIANG Yingkai, XIAO Mingwei, CHEN Qiao, LUO Chendi. Detection Method for Dragon Fruit in Natural Environment Based on Improved YOLOX [J]. Smart Agriculture, 2022, 4(3): 120-131.
[6]	WANG Hui, CHEN Ruipeng, YU Zhixue, HE Yue, ZHANG Fan, XIONG Benhai. Porphyrin and Semiconducting Single Wall Carbon Nanotubes based Semiconductor Field Effect Gas Sensor for Determination of Phytophthora Strawberries [J]. Smart Agriculture, 2022, 4(3): 143-151.
[7]	GENG Wenxuan, ZHAO Junye, RUAN Jiwei, HOU Yuehui. Comparative Study of the Regulation Effects of Artificial Intelligence-Assisted Planting Strategies on Strawberry Production in Greenhouse [J]. Smart Agriculture, 2022, 4(2): 183-193.
[8]	GUO Xiuming, ZHU Yeping, LI Shijuan, ZHANG Jie, LYU Chunyang, LIU Shengping. Scale Adaptive Small Objects Detection Method in Complex Agricultural Environment: Taking Bees as Research Object [J]. Smart Agriculture, 2022, 4(1): 140-149.
[9]	ZHANG Yu, ZHAO Chunjiang, LIN Sen, GUO Wenzhong, WEN Chaowu, LONG Jiehua. Irrigation Method and Verification of Strawberry Based on Penman-Monteith Model and Path Ranking Algorith [J]. Smart Agriculture, 2021, 3(3): 116-128.
[10]	HAO Wangli, YU Peiyan, HAO Fei, HAN Meng, HAN Jiwan, SUN Weirong, LI Fuzhong. Foxtail Millet Ear Detection Approach Based on YOLOv4 and Adaptive Anchor Box Adjustment [J]. Smart Agriculture, 2021, 3(1): 63-74.
[11]	Zhu Nanyang, Wu Hao, Yin Daheng, Wang Zhiqiang, Jiang Yongnian, Guo Ya. An improved method for estimating dissolved oxygen in crab ponds based on Long Short-Term Memory [J]. Smart Agriculture, 2019, 1(3): 67-76.