基于改进ResNet50模型的自然环境下苹果物候期识别
收稿日期: 2023-04-18
网络出版日期: 2023-05-31
基金资助
四川省财政自主创新专项项目(2022ZZCX034);人才引进培养与学科建设资金(2021XKZD01)
Apple Phenological Period Identification in Natural Environment Based on Improved ResNet50 Model
Received date: 2023-04-18
Online published: 2023-05-31
Supported by
Sichuan Provincial Financial Independent Innovation Special Project (2022ZZCX034); Talent Introduction and Discipline Construction Fund (2021XKZD01)
[目的/意义] 针对传统方法对自然环境下苹果物候期图像识别精度低、覆盖面不全等问题,提出一种基于改进ResNet50模型的苹果物候期识别方法。 [方法] 通过搭建球形摄像机获取复杂背景下的苹果图像数据集,以ResNet50作为基础模型,引入SE(Squeeze-and-Excitation Network)通道注意力机制强化对苹果图像的特征提取能力,并结合余弦退火衰减学习率的Adam优化器,实现自然环境下高原红富士苹果物候期图像的智能识别。[结果和讨论]在32,000幅苹果树图像集上进行了试验,结果表明,改进ResNet50模型对苹果物候期图像进行识别,验证集准确率达到96.35%,测试集准确率达到91.94%,平均检测时间为2.19 ms,相较于AlexNet、VGG16、ResNet18、ResNet34、ResNet101以及经典ResNet50模型,最优验证集准确率分别提升了9.63%、5.07%、5.81%、4.55%、0.96%和2.33%。 [结论] 改进ResNet50可实现对苹果物候期有效识别,该研究成果可为果园物候期识别提供参考,通过集成至果树生育期智能监测生产管理平台,实现苹果园区的智能化管控。
刘永波 , 高文波 , 何鹏 , 唐江云 , 胡亮 . 基于改进ResNet50模型的自然环境下苹果物候期识别[J]. 智慧农业, 2023 , 5(2) : 13 -22 . DOI: 10.12133/j.smartag.SA202304009
[Objective] Aiming at the problems of low accuracy and incomplete coverage of image recognition of phenological period of apple in natural environment by traditional methods, an improved ResNet50 model was proposed for phenological period recognition of apple. [Methods] With 8 kinds of phenological period images of Red Fuji apple in Sichuan plateau area as the research objects and 3 sets of spherical cameras built in apple orchard as acquisition equipment, the original data set of 9800 images of apple phenological period were obtained, labeled by fruit tree experts. Due to the different duration of each phenological period of apple, there were certain differences in the quantity of collection. In order to avoid the problem of decreasing model accuracy due to the quantity imbalance, data set was enhanced by random cropping, random rotation, horizontal flip and brightness adjustment, and the original data set was expanded to 32,000 images. It was divided into training set (25,600 images), verification set (3200 images) and test set (3200 images) in a ratio of 8:1:1. Based on the ResNet50 model, the SE (Squeeze and Excitation Network) channel attention mechanism and Adam optimizer were integrated. SE channel attention was introduced at the end of each residual module in the benchmark model to improve the model's feature extraction ability for plateau apple tree images. In order to achieve fast convergence of the model, the Adam optimizer was combined with the cosine annealing attenuation learning rate, and ImageNet was selected as the pre-training model to realize intelligent recognition of plateau Red Fuji apple phenological period under natural environment. A "Intelligent Monitoring and Production Management Platform for Fruit Tree Growth Period" has been developed using the identification model of apple tree phenology. In order to reduce the probability of model misjudgment, improve the accuracy of model recognition, and ensure precise control of the platform over the apple orchard, three sets of cameras deployed in the apple orchard were set to capture motion trajectories, and images were collected at three time a day: early, middle, and late, a total of 27 images per day were collected. The model calculated the recognition results of 27 images and takes the category with the highest number of recognition as the output result to correct the recognition rate and improve the reliability of the platform. [Results and Discussions] Experiments were carried out on 32,000 apple tree images. The results showed that when the initial learning rate of Adam optimizer was set as 0.0001, the accuracy of the test model tended to the optimal, and the loss value curve converged the fastest. When the initial learning rate was set to 0.0001 and the iteration rounds are set to 30, 50 and 70, the accuracies of the optimal verification set obtained by the model was 0.9354, 0.9635 and 0.9528, respectively. Therefore, the improved ResNet50 model selects the learning rate of 0.0001 and iteration rounds of 50 as the training parameters of the Adam optimizer. Ablation experiments showed that the accuracy of validation set and test set were increased by 0.8% and 2.99% in the ResNet50 model with increased SE attention mechanism, respectively. The validation set accuracy and test set accuracy of the ResNet50 model increased by 2.19% and 1.42%, respectively, when Adam optimizer was added. The accuracy of validation set and test set was 2.33% and 3.65%, respectively. The accuracy of validation set was 96.35%, the accuracy of test set was 91.94%, and the average detection time was 2.19 ms.Compared with the AlexNet, VGG16, ResNet18, ResNet34, and ResNet101 models, the improved ResNet50 model improved the accuracy of the optimal validation set by 9.63%, 5.07%, 5.81%, 4.55%, and 0.96%, respectively. The accuracy of the test set increased by 12.31%, 6.88%, 8.53%, 8.67%, and 5.58%, respectively. The confusion matrix experiment result showed that the overall recognition rate of the improved ResNet50 model for the phenological period of apple tree images was more than 90%, of which the accuracy rate of bud stage and dormancy stage was the lowest, and the probability of mutual misjudgment was high, and the test accuracy rates were 89.50% and 87.44% respectively. There were also a few misjudgments during the young fruit stage, fruit enlargement stage, and fruit coloring stage due to the similarity in characteristics between adjacent stages. The external characteristics of the Red Fuji apple tree were more obvious during the flowering and fruit ripening stages, and the model had the highest recognition rate for the flowering and fruit ripening stages, with test accuracy reaching 97.50% and 97.49%, respectively. [Conclusions] The improved ResNet50 can effectively identify apple phenology, and the research results can provide reference for the identification of orchard phenological period. After integration into the intelligent monitoring production management platform of fruit tree growth period, intelligent management and control of apple orchard can be realized.
Key words: apple; residual network; ResNet50; phenological period recognition; smart orchard
1 | 樊泽泽. 苹果果实检测与物候期自动识别方法的研究与实现[D]. 太原: 太原理工大学, 2020. |
FAN Z Z. Research and implementation on automatic method of apple detection and phenological period classification[D]. Taiyuan: Taiyuan University of Technology, 2020. | |
2 | 高登涛, 李丙智. 图说苹果高效栽培: 全彩版[M]. 北京: 机械工业出版社, 2018: 60-104. |
GAO D T, LI B Z. Caption efficient cultivation of apple: full color edition[M]. Beijing: China Machine Press, 2018: 60-104. | |
3 | CHEN Z Y, SU R, WANG Y L, et al. Automatic estimation of apple orchard blooming levels using the improved YOLOv5[J]. Agronomy, 2022, 12(10): ID 2483. |
4 | 李诗涛, 张王菲, 赵丽仙, 等. 基于时序PolSAR影像与决策树模型的油菜物候期识别[J]. 浙江农业学报, 2021, 33(11): 2116-2127. |
LI S T, ZHANG W F, ZHAO L X, et al. Phenological period identification of oilseed rape based on time-series PolSAR image and decision tree model[J]. Acta agriculturae zhejiangensis, 2021, 33(11): 2116-2127. | |
5 | 王志毅, 王嘉佩, 杜爱军, 等. 基于深度学习的自动判别茶叶生长的物候期模型的建立[J]. 气象科技进展, 2021, 11(2): 119-120, 137. |
WANG Z Y, WANG J P, DU A J, et al. A deep-learning based model used to automaticly identification white tea's phenological period[J]. Advances in meteorological science and technology, 2021, 11(2): 119-120, 137. | |
6 | TAN S Y, LU H H, YU J, et al. In-field rice panicles detection and growth stages recognition based on RiceRes2Net[J]. Computers and electronics in agriculture, 2023, 206: ID 107704. |
7 | LI Y, LIU H B, WEI J L, et al. Research on winter wheat growth stages recognition based on mobile edge computing[J]. 2023, 13(3): 534-550. |
8 | AGUIAR A S, MAGALH?ES S A, DOS SANTOS F N, et al. Grape bunch detection at different growth stages using deep learning quantized models[J]. Agronomy, 2021, 11(9): ID 1890. |
9 | TIAN Y N, YANG G D, WANG Z, et al. Apple detection during different growth stages in orchards using the improved YOLOv3 model[J]. Computers and electronics in agriculture, 2019, 157: 417-426. |
10 | 许增, 王志伟, 胡桃花, 等. 改进的轻量级YOLO在苹果物候期自动观测中的研究[J]. 计算机工程与设计, 2021, 42(12): 3478-3484. |
XU Z, WANG Z W, HU T H, et al. Improved lightweight YOLO in automatic observation of apple phenology[J]. Computer engineering and design, 2021, 42(12): 3478-3484. | |
11 | 傅隆生, 宋珍珍, ZHANG X, 等. 深度学习方法在农业信息中的研究进展与应用现状[J]. 中国农业大学学报, 2020, 25(2): 105-120. |
FU L S, SONG Z Z, ZHANG X, et al. Applications and research progress of deep learning in agriculture[J]. Journal of China agricultural university, 2020, 25(2): 105-120. | |
12 | HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2016: 770-778. |
13 | LI X, RAI L. Apple leaf disease identification and classification using ResNet models[C]// 2020 IEEE 3rd International Conference on Electronic Information and Communication Technology (ICEICT). Piscataway, NJ, USA: IEEE, 2021: 738-742. |
14 | LI B Q, HE Y Y. An improved ResNet based on the adjustable shortcut connections[J]. IEEE access, 2018, 6: 18967-18974. |
15 | 杨春兰, 朱鹏飞, 许成祥. 融合注意力机制的淡水鱼类识别方法[J]. 西南民族大学学报(自然科学版), 2023, 49(1): 83-93. |
YANG C L, ZHU P F, XU C X. Freshwater fish identification method incorporating attention mechanism[J]. Journal of Southwest minzu university (natural science edition), 2023, 49(1): 83-93. | |
16 | 卓力, 袁帅, 李嘉锋. 基于ResNet50和通道注意力机制的行人多属性协同识别方法[J]. 测控技术, 2022, 41(8): 1-8, 15. |
ZHUO L, YUAN S, LI J F. Pedestrian multi-attribute collaborative recognition method based on ResNet50 and channel attention mechanism[J]. Measurement & control technology, 2022, 41(8): 1-8, 15. | |
17 | 刘永波, 黄强, 高文波, 等. 融合BERT-WWM和注意力机制的茶叶知识图谱构建[J]. 西南农业学报, 2022, 35(12): 2912-2921. |
LIU Y B, HUANG Q, GAO W B, et al. Construction of knowledge graph of integrating BERT-WWM and attention mechanism[J]. Southwest China journal of agricultural sciences, 2022, 35(12): 2912-2921. | |
18 | HU J, SHEN L, ALBANIE S, et al. Squeeze-and-excitation networks[J]. IEEE transactions on pattern analysis and machine intelligence, 2020, 42(8): 2011-2023. |
19 | HE J, JIANG D. Fully automatic model based on SE-ResNet for bone age assessment[J]. IEEE access, 2021, 9: 62460-62466. |
20 | 龙洁花, 郭文忠, 林森, 等. 改进YOLOv4的温室环境下草莓生育期识别方法[J]. 智慧农业(中英文), 2021, 3(4): 99-110. |
LONG J H, GUO W Z, LIN S, et al. Strawberry growth period recognition method under greenhouse environment based on improved YOLOv4[J]. Smart agriculture, 2021, 3(4): 99-110. | |
21 | 万鹏, 赵竣威, 朱明, 等. 基于改进Res Net50模型的大宗淡水鱼种类识别方法[J]. 农业工程学报, 2021, 37(12): 159-168. |
WAN P, ZHAO J W, ZHU M, et al. Freshwater fish species identification method based on improved Res Net50 model[J]. Transactions of the Chinese society of agricultural engineering, 2021, 37(12): 159-168. | |
22 | KINGMA D P, BA J. Adam: A method for stochastic optimization[EB/OL]. arXiv: , 2014. |
23 | DU H, WANG W, WANG X R, et al. Autonomous landing scene recognition based on transfer learning for drones[J]. Journal of systems engineering and electronics, 2023, 34(1): 28-35. |
24 | 刘翱宇, 吴云志, 朱小宁, 等. 基于深度残差网络的玉米病害识别[J]. 江苏农业学报, 2021, 37(1): 67-74. |
LIU A Y, WU Y Z, ZHU X N, et al. Corn disease recognition based on deep residual network[J]. Jiangsu journal of agricultural sciences, 2021, 37(1): 67-74. |
/
〈 |
|
〉 |