Welcome to Smart Agriculture 中文
Topic--Machine Vision and Agricultural Intelligent Perception

Desert Plant Recognition Method Under Natural Background Incorporating Transfer Learning and Ensemble Learning

  • WANG Yapeng ,
  • CAO Shanshan ,
  • LI Quansheng ,
  • SUN Wei
Expand
  • 1.Computer and Information Engineering College, Xinjiang Agricultural University, Urumqi 830052, China
    2.Agricultural Information Institute, Chinese Academy of Agricultural Sciences, Beijing 100081, China
    3.National Agriculture Science Data Center, Beijing 100081, China
WANG Yapeng, E-mail:1033453569@qq.com
SUN Wei, E-mail:sunwei02@caas.cn

Received date: 2023-05-03

  Online published: 2023-07-15

Supported by

National Natural Science Foundation Project (32271880)

Abstract

[Objective] Desert vegetation is an indispensable part of desert ecosystems, and its conservation and restoration are crucial. Accurate identification of desert plants is an indispensable task, and is the basis of desert ecological research and conservation. The complex growth environment caused by light, soil, shadow and other vegetation increases the recognition difficulty, and the generalization ability is poor and the recognition accuracy is not guaranteed. The rapid development of modern technology provides new opportunities for plant identification and classification. By using intelligent identification algorithms, field investigators can be effectively assisted in desert plant identification and classification, thus improve efficiency and accuracy, while reduce the associated human and material costs. [Methods] In this research, the following works were carried out for the recognition of desert plant: Firstly, a training dataset of deep learning model of desert plant images in the arid and semi-arid region of Xinjiang was constructed to provide data resources and basic support for the classification and recognition of desert plant images.The desert plant image data was collected in Changji and Tacheng region from the end of September 2021 and July to August 2022, and named DPlants50. The dataset contains 50 plant species in 13 families and 43 genera with a total of 12,507 images, and the number of images for each plant ranges from 183 to 339. Secondly, a migration integration learning-based algorithm for desert plant image recognition was proposed, which could effectively improve the recognition accuracy. Taking the EfficientNet B0-B4 network as the base network, the ImageNet dataset was pre-trained by migration learning, and then an integrated learning strategy was adopted combining Bagging and Stacking, which was divided into two layers. The first layer introduced K-fold cross-validation to divide the dataset and trained K sub-models by borrowing the Stacking method. Considering that the output features of each model were the same in this study, the second layer used Bagging to integrate the output features of the first layer model by voting method, and the difference was that the same sub-models and K sub-models were compared to select the better model, so as to build the integrated model, reduce the model bias and variance, and improve the recognition performance of the model. For 50 types of desert plants, 20% of the data was divided as the test set, and the remaining 5 fold cross validation was used to divide the dataset, then can use DPi(i=1,2,…,5) represents each training or validation set. Based on the pre trained EfficientNet B0-B4 network, training and validation were conducted on 5 data subsets. Finally, the model was integrated using soft voting, hard voting, and weighted voting methods, and tested on the test set. [Results and Discussions] The results showed that the Top-1 accuracy of the single sub-model based on EfficientNet B0 network was 92.26%~93.35%, the accuracy of the Ensemble-Soft model with soft voting, the Ensemble-Hard model with hard voting and the Ensemble-Weight model integrated by weighted voting method were 93.63%, 93.55% and 93.67%, F1 Score and accuracy were comparable, the accuracy and F1 Score of Ensemble-Weight model integrated by weighted voting method were not significantly improved compared with Ensemble-Soft model and Ensemble-hard model, but it showed that the effect of weighted voting method proposed in this study was better than both of them. The three integrated models demonstrate no noteworthy enhancements in accuracy and F1 Score when juxtaposed with the five sub-models. This observation results suggests that the homogeneity among the models constrains the effectiveness of the voting method strategy. Moreover, the recognition effects heavily hinges on the performance of the EfficientNet B0-DP5 model. Therefore, the inclusion of networks with more pronounced differences was considered as sub-models. A single sub-model based on EfficientNet B0-B4 network had the highest Top-1 accuracy of 96.65% and F1 Score of 96.71%, while Ensemble-Soft model, Ensemble-Hard model and Ensemble-Weight model got the accuracy of 99.07%, 98.91% and 99.23%, which further improved the accuracy compared to the single sub-model, and the F1 Score was basically the same as the accuracy rate, and the model performance was significant. The model integrated by the weighted voting method also improved accuracy and F1 Score for both soft and hard voting, with significant model performance and better recognition, again indicating that the weighted voting method was more effective than the other two. Validated on the publicly available dataset Oxford Flowers102, the three integrated models improved the accuracy and F1 Score of the three sub-models compared to the five sub-models by a maximum of 4.56% and 5.05%, and a minimum of 1.94% and 2.29%, which proved that the migration and integration learning strategy proposed in this paper could effectively improve the model performances. [Conclusions] In this study, a method to recognize desert plant images in natural context by integrating migration learning and integration learning was proposed, which could improve the recognition accuracy of desert plants up to 99.23% and provide a solution to the problems of low accuracy, model robustness and weak generalization of plant images in real field environment. After transferring to the server through the cloud, it can realize the accurate recognition of desert plants and serve the scenes of field investigation, teaching science and scientific experiment.

Cite this article

WANG Yapeng , CAO Shanshan , LI Quansheng , SUN Wei . Desert Plant Recognition Method Under Natural Background Incorporating Transfer Learning and Ensemble Learning[J]. Smart Agriculture, 2023 , 5(2) : 93 -103 . DOI: 10.12133/j.smartag.SA202305001

References

1 宋智芳. 伊犁绢蒿荒漠草地植被特征对放牧干扰的响应[D]. 乌鲁木齐: 新疆农业大学, 2018.
  SONG Z F. Response of Seriphidium transiliense vegetation characteristics to grazing disturance in desert grasslands[D]. Urumqi: Xinjiang Agricultural University, 2018.
2 滕迎凤. 宁夏沙湖自然保护区植物多样性研究[D]. 银川: 宁夏大学, 2013.
  TENG Y F. Studies on diversity of the plants in Shahu nature reserve, Ningxia, China[D]. Yinchuan: Ningxia University, 2013.
3 燕辉. 西北旱区两种典型沙生植物对盐胁迫响应的研究[D]. 杨凌: 西北农林科技大学, 2012.
  YAN H. The response of two representative desert shrubs to salt stress in northwest arid region[D]. Yangling: Northwest A & F University, 2012.
4 何恒斌. 沙冬青群落及其根瘤菌的研究[D]. 北京: 北京林业大学, 2008.
  HE H B. Studies on communities and rhizoibum of Ammopiptanthus mongolicus (maxim.)[D]. Beijing: Beijing Forestry University, 2008.
5 LECUN Y, BENGIO Y, HINTON G. Deep learning[J]. Nature, 2015, 521(7553): 436-444.
6 GOODFELLOW I, BENGIO Y, COURVILLE A. Deep learning[M]. Cambridge, Massachusetts: The MIT Press, 2016.
7 KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90.
8 JEON W S, RHEE S Y. Plant leaf recognition using a convolution neural network[J]. The international journal of fuzzy logic and intelligent systems, 2017, 17(1): 26-34.
9 LEE S H, CHAN C S, WILKIN P, et al. Deep-plant: Plant identification with convolutional neural networks[C]// 2015 IEEE International Conference on Image Processing (ICIP). Piscataway, NJ, USA: IEEE, 2015: 452-456.
10 韩斌, 曾松伟. 基于多特征融合和卷积神经网络的植物叶片识别[J]. 计算机科学, 2021, 48(S1): 113-117.
  HAN B, ZENG S W. Plant leaf image recognition based on multi-feature integration and convolutional neural network[J]. Computer science, 2021, 48(S1): 113-117.
11 金莉婷. 基于卷积神经网络的复杂背景植物图像识别研究[D]. 兰州: 兰州交通大学, 2020.
  JIN L T. Research on plant image recognition with complex background based on convolution neural network[D]. Lanzhou: Lanzhou Jiaotong University, 2020.
12 冯海林, 胡明越, 杨垠晖, 等. 基于树木整体图像和集成迁移学习的树种识别[J]. 农业机械学报, 2019, 50(8): 235-242, 279.
  FENG H L, HU M Y, YANG Y H, et al. Tree species recognition based on overall tree image and ensemble of transfer learning[J]. Transactions of the Chinese society for agricultural machinery, 2019, 50(8): 235-242, 279.
13 宋晓宇, 金莉婷, 赵阳, 等. 基于有效区域筛选的复杂背景植物图像识别方法[J]. 激光与光电子学进展, 2020, 57(4): 181-191.
  SONG X Y, JIN L T, ZHAO Y, et al. Plant image recognition with complex background based on effective region screening[J]. Laser & optoelectronics progress, 2020, 57(4): 181-191.
14 ZHOU J, LI J X, WANG C S, et al. A vegetable disease recognition model for complex background based on region proposal and progressive learning[J]. Computers and electronics in agriculture, 2021, 184: ID 106101.
15 LI J C, SUN S D, JIANG H R, et al. Image recognition and empirical application of desert plant species based on convolutional neural network[J]. Journal of arid land, 2022, 14(12): 1440-1455.
16 曹香滢, 孙卫民, 朱悠翔, 等. 基于科优先策略的植物图像识别[J]. 计算机应用, 2018, 38(11): 3241-3245.
  CAO X Y, SUN W M, ZHU Y X, et al. Plant image recoginiton based on family priority strategy[J]. Journal of computer applications, 2018, 38(11): 3241-3245.
17 郭晓丽. 基于全卷积神经网络的植物图像分割算法研究与实现[D]. 呼和浩特: 内蒙古大学, 2021.
  GUO X L. Research and implementation on plant image segementation algorithm based on neural network[D]. Hohhot: Inner Mongolia University, 2021.
18 RAGHU M, POOLE B, KLEINBERG J, et al. On the expressive power of deep neural networks[C]// Proceedings of the 34th International Conference on Machine Learning -Volume 70. New York, USA: ACM, 2017: 2847-2854.
19 ZAGORUYKO S, KOMODAKIS N. Wide residual networks[EB/OL]. arXiv: , 2016.
20 TAN M, LE Q. EfficientNet: Rethinking model scaling for convolutional neural networks[EB/OL]. International conference on machine learning. arXiv:, 2019.
21 PAN S J, YANG Q. A survey on transfer learning[J]. IEEE transactions on knowledge and data engineering, 2010, 22(10): 1345-1359.
22 DONG X B, YU Z W, CAO W M, et al. A survey on ensemble learning[J]. Frontiers of computer science, 2020, 14(2): 241-258.
23 WANG B, PINEAU J. Online bagging and boosting for imbalanced data streams[J]. IEEE transactions on knowledge and data engineering, 2016, 28(12): 3353-3366.
24 HUI Y, MEI X S, JIANG G D, et al. Milling tool wear state recognition by vibration signal using a stacked generalization ensemble model[J]. Shock and vibration, 2019, 2019: 1-16.
25 ANDIOJAYA A, DEMIRHAN H. A bagging algorithm for the imputation of missing values in time series[J]. Expert systems with applications, 2019, 129: 10-26.
26 FIELDING A H, BELL J F. A review of methods for the assessment of prediction errors in conservation presence/absence models[J]. Environmental conservation, 1997, 24(1): 38-49.
27 高宏元, 高新华, 冯琦胜, 等. 基于深度学习的天然草地植物物种识别方法[J]. 草业科学, 2020, 37(9): 1931-1939.
  GAO H Y, GAO X H, FENG Q S, et al. Approach to plant species identification in natural grasslands based on deep learning[J]. Pratacultural science, 2020, 37(9): 1931-1939.
28 彭文, 兰玉彬, 岳学军, 等. 基于深度卷积神经网络的水稻田杂草识别研究[J]. 华南农业大学学报, 2020, 41(6): 75-81.
  PENG W, LAN Y B, YUE X J, et al. Research on paddy weed recognition based on deep convolutional neural network[J]. Journal of South China agricultural university, 2020, 41(6): 75-81.
29 陈淑君, 周永霞, 方勇军. 基于整体外观特征的植物种类识别研究[J]. 计算机应用与软件, 2017, 34(9): 222-227.
  CHEN S J, ZHOU Y X, FANG Y J. The plant species recognition based on the whole appearanc features[J]. Computer applications and software, 2017, 34(9): 222-227.
Outlines

/