欢迎您访问《智慧农业(中英文)》官方网站! English

Smart Agriculture ›› 2023, Vol. 5 ›› Issue (3): 96-109.doi: 10.12133/j.smartag.SA202308003

• 专刊--作物信息监测技术 • 上一篇    下一篇

融合改进UNet和迁移学习的棉花根系图像分割方法

唐辉1(), 王铭2, 于秋实1, 张佳茜1, 刘连涛3, 王楠1()   

  1. 1. 河北农业大学 机电工程学院, 河北 保定 071001, 中国
    2. 河北省教育考试院, 河北 石家庄 050091, 中国
    3. 河北农业大学 农学院, 河北 保定 071001, 中国
  • 收稿日期:2023-07-28 出版日期:2023-09-30
  • 基金资助:
    河北省教育厅青年拔尖人才计划项目(BJ2021058); 中央引导地方科技发展资金项目(236Z7402G); 华北作物改良与调控国家重点实验室自主课题(NCCIR2021ZZ-23)
  • 作者简介:
    唐 辉,研究方向为深度学习、根系表型。E-mail:
  • 通信作者:
    王 楠,博士,副教授,研究方向为表型组学、图像处理。E-mail:

Root Image Segmentation Method Based on Improved UNet and Transfer Learning

TANG Hui1(), WANG Ming2, YU Qiushi1, ZHANG Jiaxi1, LIU Liantao3, WANG Nan1()   

  1. 1. College of mechanical and electrical engineering, Hebei Agricultural University, Baoding 071001, China
    2. Hebei Education Examinations Authority, Shijiazhuang 050091, China
    3. College of agronomy, Hebei Agricultural University, Baoding 071001, China
  • Received:2023-07-28 Online:2023-09-30
  • corresponding author: WANG  Nan, E-mail:
  • About author:TANG Hui, E-mail:318901249@qq.com
  • Supported by:
    Top-notch Talent Plan Program of the Education Department of Hebei Province(BJ2021058); Central Guiding Local Science and Technology Development Fund Projects(236Z7402G); State Key Laboratory of North China Crop Improvement and Regulation(NCCIR2021ZZ-23)

摘要:

[目的/意义] 根系是植物组成的重要部分,其生长发育至关重要。根系图像分割是根系表型分析的重要方法,受限于图像质量、复杂土壤环境、低效传统方法,根系图像分割存在一定挑战。 [方法] 为提高根系图像分割的准确性和鲁棒性,本研究以UNet模型为基础,提出了一种多尺度特征提取根系分割算法,并结合数据增强和迁移学习进一步提高改进UNet模型的泛化性和通用性。首先,获取棉花根系单一数据集和开源多作物混合数据集,基于单一数据集的消融试验测试多尺度特征提取模块(Conv_2+Add)的有效性,与UNet、PSPNet、SegNet、DeeplabV3Plus算法对比验证其优势。基于混合数据集验证改进算法(UNet+Conv_2+Add)在迁移学习的优势。 [结果和讨论] UNet+Conv_2+Add相比其他算法(UNet、PSPNet、SegNet、DeeplabV3Plus),mIoU、mRecall和根系F1调和平均值分别为81.62%、86.90%和78.39%。UNet+Conv_2+Add算法的迁移学习相比于普通训练在根系的交并比(Intersection over Union,IoU)值提升1.25%,根系的Recall值提升1.79%,F1调和平均值提升0.92%,且模型的整体收敛速度快。 [结论] 本研究采用的多尺度特征提取策略能准确、高效地分割根系,为作物根系表型研究提供重要的研究基础。

关键词: 深度学习, 根系图像分割, UNet, 多尺度特征, 迁移学习

Abstract:

[Objective] The root system is an important component of plant composition, and its growth and development are crucial for plants. Root image segmentation is an important method for obtaining root phenotype information and analyzing root growth patterns. Research on root image segmentation still faces difficulties, because of the noise and image quality limitations, the intricate and diverse soil environment, and the ineffectiveness of conventional techniques. This paper proposed a multi-scale feature extraction root segmentation algorithm that combined data augmentation and transfer learning to enhance the generalization and universality of the root image segmentation models in order to increase the speed, accuracy, and resilience of root image segmentation. [Methods] Firstly, the experimental datasets were divided into a single dataset and a mixed dataset. The single dataset acquisition was obtained from the experimental station of Hebei Agricultural University in Baoding city. Additionally, a self-made RhizoPot device was used to collect images with a resolution pixels of 10,200×14,039, resulting in a total of 600 images. In this experiment, 100 sheets were randomly selected to be manually labeled using Adobe Photoshop CC2020 and segmented into resolution pixels of 768×768, and divided into training, validation, and test sets according to 7:2:1. To increase the number of experimental samples, an open source multi-crop mixed dataset was obtained in the network as a supplement, and it was reclassified into training, validation, and testing sets. The model was trained using the data augmentation strategy, which involved performing data augmentation operations at a set probability of 0.3 during the image reading phase, and each method did not affect the other. When the probability was less than 0.3, changes would be made to the image. Specific data augmentation methods included changing image attributes, randomly cropping, rotating, and flipping those images. The UNet structure was improved by designing eight different multi-scale image feature extraction modules. The module structure mainly included two aspects: Image convolution and feature fusion. The convolution improvement included convolutional block attention module (CBAM), depthwise separable convolution (DP Conv), and convolution (Conv). In terms of feature fusion methods, improvements could be divided into concatenation and addition. Subsequently, ablation tests were conducted based on a single dataset, data augmentation, and random loading of model weights, and the optimal multi-scale feature extraction module was selected and compared with the original UNet. Similarly, a single dataset, data augmentation, and random loading of model weights were used to compare and validate the advantages of the improved model with the PSPNet, SegNet, and DeeplabV3Plus algorithms. The improved model used pre-trained weights from a single dataset to load and train the model based on mixed datasets and data augmentation, further improving the model's generalization ability and root segmentation ability. [Results and Discussions] The results of the ablation tests indicated that Conv_ 2+Add was the best improved algorithm. Compared to the original UNet, the mIoU, mRecall, and root F1 values of the model increased by 0.37%, 0.99%, and 0.56%, respectively. And, comparative experiments indicate Unet+Conv_2+Add model was superior to the PSPNet, SegNet, and DeeplabV3Plus models, with the best evaluation results. And the values of mIoU, mRecall, and the harmonic average of root F1 were 81.62%, 86.90%, and 77.97%, respectively. The actual segmented images obtained by the improved model were more finely processed at the root boundary compared to other models. However, for roots with deep color and low contrast with soil particles, the improved model could only achieve root recognition and the recognition was sparse, sacrificing a certain amount of information extraction ability. This study used the root phenotype evaluation software Rhizovision to analyze the root images of the Unet+Conv_2+Add improved model, PSPNet, SegNet, and DeeplabV3Plu, respectively, to obtain the values of the four root phenotypes (total root length, average diameter, surface area, and capacity), and the results showed that the average diameter and surface area indicator values of the improved model, Unet+Conv_2+Add had the smallest differences from the manually labeled indicator values and the SegNet indicator values for the two indicators. Total root length and volume were the closest to those of the manual labeling. The results of transfer learning experiments proved that compared with ordinary training, the transfer training of the improved model UNet+Conv_2+Add increased the IoU value of the root system by 1.25%. The Recall value of the root system was increased by 1.79%, and the harmonic average value of F1 was increased by 0.92%. Moreover, the overall convergence speed of the model was fast. Compared with regular training, the transfer training of the original UNet improved the root IoU by 0.29%, the root Recall by 0.83%, and the root F1 value by 0.21%, which indirectly confirmed the effectiveness of transfer learning. [Conclusions] The multi-scale feature extraction strategy proposed in this study can accurately and efficiently segment roots, and further improve the model's generalization ability using transfer learning methods, providing an important research foundation for crop root phenotype research.

Key words: deep learning, root image segmentation, UNet, multi-scale characteristics, transfer learning