Extraction of Potato Plant Phenotypic Parameters Based on Multi-Source Data
Received date: 2023-02-28
Online published: 2023-05-11
Supported by
National Natural Science Foundation of China (U21A20205)
Crops have diverse structures and complex growth environments. RGB image data can reflect the texture and color features of plants accurately, while 3D data contains information about crop volume. The combination of RGB image and 3D point cloud data can achieve the extraction of two-dimensional and three-dimensional phenotypic parameters of crops, which is of great significance for the research of phenomics methods. In this study, potatoe plants were chosen as the research subject, and RGB cameras and laser scanners were used to collect 50 potato RGB images and 3D laser point cloud data. The segmentation accuracy of four deep learning semantic segmentation methods, OCRNet, UpNet, PaNet, and DeepLab v3+, were compared and analyzed for the RGB images. OCRNet, which demonstrated higher accuracy, was used to perform semantic segmentation on top-view RGB images of potatoes. Mean shift clustering algorithm was optimized for laser point cloud data processing, and single-plant segmentation of laser point cloud data was completed. Stem and leaf segmentation of single-plant potato point cloud data were accurately performed using Euclidean clustering and K-Means clustering algorithms. In addition, a strategy was proposed to establish a one-to-one correspondence between RGB images and point clouds of single-plant potatoes using pot numbering. 8 2D phenotypic parameters and 10 3D phenotypic parameters, including maximum width, perimeter, area, plant height, volume, leaf length, and leaf width, etc., were extracted from RGB images and laser point clouds, respectively. Finally, the accuracy of three representative and easily measurable phenotypic parameters, leaf number, plant height, and maximum width were evaluated. The mean absolute percentage errors (MAPE) were 8.6%, 8.3% and 6.0%, respectively, while the root mean square errors (RMSE) were 1.371 pieces, 3.2 cm and 1.86 cm, respectively, and the determination coefficients (R2) were 0.93, 0.95 and 0.91, respectively. The research results indicated that the extracted phenotype parameters can accurately and efficiently reflect the growth status of potatoes. Combining the RGB image data of potatoes with three-dimensional laser point cloud data can fully exploit the advantages of the rich texture and color characteristics of RGB images and the volumetric information provided by three-dimensional point clouds, achieving non-destructive, efficient, and high-precision extraction of two-dimensional and three-dimensional phenotype parameters of potato plants. The achievements of this study could not only provide important technical support for the cultivation and breeding of potatoes but also provide strong support for phenotype-based research.
HU Songtao , ZHAI Ruifang , WANG Yinghua , LIU Zhi , ZHU Jianzhong , REN He , YANG Wanneng , SONG Peng . Extraction of Potato Plant Phenotypic Parameters Based on Multi-Source Data[J]. Smart Agriculture, 2023 , 5(1) : 132 -145 . DOI: 10.12133/j.smartag.SA202302009
1 | 翁杨, 曾睿, 吴陈铭, 等. 基于深度学习的农业植物表型研究综述[J]. 中国科学(生命科学), 2019, 49(6): 698-716. |
WENG Y, ZENG R, WU C M, et al. A survey on deep-learning-based plant phenotype research in agriculture[J]. Scientia sinica (vitae), 2019, 49(6): 698-716. | |
2 | GERMAIN C, ROUSSEAUD R, GRENIER G. Non destructive counting of wheatear with picture analysis[C]//Fifth International Conference on Image Processing and its Applications. London, UK: IET, 2002: 435-439. |
3 | 王传宇, 郭新宇, 杜建军, 等. 田间环境下玉米叶片图像分割方法及装置: CN106296662B[P]. 2019-07-02. |
WANG C Y, GUO X Y, DU J J, et al. Corn leaf image segmentation method and apparatus in field environment: CN106296662B[P]. 2019-07-02. | |
4 | LORESCO P J M, VALENZUELA I C, DADIOS E P. Color space analysis using KNN for lettuce crop stages identification in smart farm setup[C]// TENCON 2018—2018 IEEE Region 10 Conference. Piscataway, NJ, USA: IEEE, 2019: 2040-2044. |
5 | GRINBLAT G L, UZAL L C, LARESE M G, et al. Deep learning for plant identification using vein morphological patterns[J]. Computers and electronics in agriculture, 2016, 127: 418-424. |
6 | LEE S H, CHAN C S, WILKIN P, et al. Deep-plant: Plant identification with convolutional neural networks[C]//2015 IEEE International Conference on Image Processing (ICIP). Piscataway, NJ, USA: IEEE, 2015: 452-456. |
7 | FAN Z Q, SUN N, QIU Q, et al. A high-throughput phenotyping robot for measuring stalk diameters of maize crops[C]// 2021 IEEE 11th Annual International Conference on CYBER Technology in Automation, Control, and Intelligent Systems (CYBER). Piscataway, NJ, USA: IEEE, 2021: 128-133. |
8 | 王晓婷, 赵展, 王阳, 等. 基于改进Mask R-CNN的植物表型智能检测算法[J]. 中国农机化学报, 2022, 43(8): 151-157. |
WANG X T, ZHAO Z, WANG Y, et al. Intelligent detection algorithm of plant phenotype based on improved Mask R: CNN[J]. Journal of Chinese agricultural mechanization, 2022, 43(8): 151-157. | |
9 | ZHANG J Y, ZHAO X L, CHEN Z, et al. A review of deep learning-based semantic segmentation for point cloud[J]. IEEE access, 2019, 7: 179118-179133. |
10 | 周静静, 郭新宇, 吴升, 等. 基于多视角图像的植物三维重建研究进展[J]. 中国农业科技导报, 2019, 21(2): 9-18. |
ZHOU J J, GUO X Y, WU S, et al. Research progress on plant three-dimensional reconstruction based on multi-view images[J]. Journal of agricultural science and technology, 2019, 21(2): 9-18. | |
11 | WANG Y J, WEN W L, WU S, et al. Maize plant phenotyping: Comparing 3D laser scanning, multi-view stereo reconstruction, and 3D digitizing estimates[J]. Remote sensing, 2018, 11(1): ID 63. |
12 | WANG Y W, CHEN Y F. Non-destructive measurement of three-dimensional plants based on point cloud[J]. Plants (basel, Switzerland), 2020, 9(5): 571. |
13 | 阳旭, 胡松涛, 王应华, 等. 利用多时序激光点云数据提取棉花表型参数方法[J]. 智慧农业(中英文), 2021, 3(1): 51-62. |
YANG X, HU S T, WANG Y H, et al. Cotton phenotypic trait extraction using multi-temporal laser point clouds[J]. Smart agriculture, 2021, 3(1): 51-62. | |
14 | 屈冬玉, 谢开云, 金黎平, 等. 中国马铃薯产业发展与食物安全[J]. 中国农业科学, 2005, 38(2): 358-362. |
QU D Y, XIE K Y, JIN L P, et al. Development of potato industry and food security in China[J]. Scientia agricultura Sinica, 2005, 38(2): 358-362. | |
15 | 张兆国, 张振东, 李加念, 等. 采用改进YOLOv4模型检测复杂环境下马铃薯[J]. 农业工程学报, 2021, 37(22): 170-178. |
ZHANG Z G, ZHANG Z D, LI J N, et al. Potato detection in complex environment based on improved YOLOv4 model[J]. Transactions of the Chinese society of agricultural engineering, 2021, 37(22): 170-178. | |
16 | 赵越, 赵辉, 姜永成, 等. 基于深度学习的马铃薯叶片病害检测方法[J]. 中国农机化学报, 2022, 43(10): 183-189. |
ZHAO Y, ZHAO H, JIANG Y C, et al. Detection method of potato leaf diseases based on deep learning[J]. Journal of Chinese agricultural mechanization, 2022, 43(10): 183-189. | |
17 | 谭彧, 田芳, 刘星星, 等. 基于RGB-D相机的马铃薯图像采集装置及芽眼识别和定位方法: CN108830272A[P]. 2018-11-16. |
TAN Y, TIAN F, LIU X X, et al. Potato image acquisition device based on RGB-D cameras and method for identifying and locating bud eye: CN108830272A[P]. 2018-11-16. | |
18 | 冯慧, 杨万能, 叶军立, 等. 一种用于盆栽水稻高光谱自动成像的双通道自动输送装置及控制方法: CN107720213B[P]. 2020-10-27. |
FENG H, YANG W N, YE J L, et al. Automatic two-channel conveying device for automatic hyperspectral imaging of rice plants and control method: CN107720213B[P]. 2020-10-27. | |
19 | YUAN Y H, CHEN X K, CHEN X L, et al. Segmentation transformer: Object-contextual representations for semantic segmentation[J/OL]. arXiv: , 2019. |
20 | 宋立鹏. 室外场景三维点云数据的分割与分类[D]. 大连: 大连理工大学, 2016. |
SONG L P. Segmentation and classification of 3D point cloud data in outdoor scenes[D]. Dalian: Dalian University of Technology, 2016. | |
21 | RUSU R B. Semantic 3D object maps for everyday manipulation in human living environments[J]. KI-Künstliche Intelligenz, 2010, 24: 345-348. |
22 | HARTIGAN J A, WONG M A. A K-means clustering algorithm[J]. Journal of the royal statistical society: Series C (applied statistics), 1979, 28(1): 100-108. |
23 | 方敏, 徐俊艳, 王建平, 等. 一种新的文本图像二值化方法[J]. 合肥工业大学学报(自然科学版), 2001, 24(2): 166-169 |
FANG M, XU J Y, WANG J P, et al. A new binarization algorithm for document image[J]. Journal of Hefei university of technology (Natural Science), 2001, 24(2): 166-169 | |
24 | LECUN Y, CORTES C, BURGES C J C. THE MNIST DATABASE of handwritten digits [EB/OL]. (1996-11-08) [2023-01-10]. |
25 | KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Commun ACM, 2017, 60(6): 84-90. |
26 | 熊雄. 基于深度学习的大田水稻稻穗分割及无损产量预估研究[D]. 武汉: 华中科技大学, 2018. |
XIONG X. Study on rice ear segmentation and lossless yield prediction of field rice based on deep learning[D].Wuhan: Huazhong University of Science and Technology, 2018. | |
27 | 杨万里, 段凌凤, 杨万能. 基于深度学习的水稻表型特征提取和穗质量预测研究[J]. 华中农业大学学报, 2021, 40(1): 227-235. |
YANG W L, DUAN L F, YANG W N. Deep learning-based extraction of rice phenotypic characteristics and prediction of rice panicle weight[J]. Journal of Huazhong agricultural university, 2021, 40(1): 227-235. | |
28 | CHEN L C, ZHU Y K, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]// European conference on computer vision. Berlin, German: Springer, 2018: 833-851. |
29 | XIAO T T, LIU Y C, ZHOU B L, et al. Unified perceptual parsing for scene understanding[C]// European Conference on Computer Vision. Berlin, German: Springer, 2018: 432-448. |
30 | LIU S, QI L, QIN H F, et al. Path aggregation network for instance segmentation[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, USA: IEEE, 2018: 8759-8768. |
/
〈 |
|
〉 |