基于多源数据的马铃薯植株表型参数提取
收稿日期: 2023-02-28
网络出版日期: 2023-05-11
基金资助
国家自然科学基金(U21A20205)
Extraction of Potato Plant Phenotypic Parameters Based on Multi-Source Data
Received date: 2023-02-28
Online published: 2023-05-11
Supported by
National Natural Science Foundation of China (U21A20205)
作物具有结构多样、生长环境复杂等特征。RGB图像数据能真实地反映植株的纹理特征与颜色特征,三维点云数据包含了作物的体量信息。将RGB图像和三维点云数据结合,实现作物的二维和三维表型参数提取,对表型组学的方法研究具有重要意义。本研究以马铃薯为研究对象,使用RGB相机和激光扫描仪分别采集了50个马铃薯的RGB图像与三维激光点云数据。对比了OCRNet,UpNet,PaNet和DeepLab v3+四种深度学习语义分割方法的分割精度,并选择精度较高的OCRNet网络实现马铃薯顶视图像的语义分割。优化了Mean shift聚类算法流程,完成了马铃薯植株激光点云的单株分割,并结合欧式聚类和K-Means聚类算法对单株马铃薯植株点云的茎和叶进行准确地分割。同时,提出一种利用编号建立马铃薯单株RGB图像和激光点云间一一对应关系的策略,并以此为基础分别从RGB图像与激光点云中提取同一马铃薯植株包括最大宽度、周长、面积、株高、体积、叶长与叶宽在内的8个二维表型参数与10个三维表型参数。最后,选择了比较具有代表性、易测量的叶片数、株高、最大宽度三个表型参数进行精度评估,平均绝对百分比误差(Mean Absolute Percentage Error,MAPE)分别为8.6%、8.3%和6.0%,均方根误差(Root Mean Square Error,RMSE)分别为1.371片、3.2 cm和1.86 cm,决定系数R2分别为0.93、0.95和0.91。精度评估的结果表明,所提取的表型参数能够准确、高效地反映马铃薯的生长状态,将马铃薯的RGB影像数据与三维激光点云数据相结合,能够充分发挥RGB图像纹理颜色特征丰富、三维点云能够提供体量信息的优势,实现马铃薯植株二维与三维表型参数高精度、非破坏性的提取。本研究成果不仅可以为马铃薯的种植和育种提供重要的技术支持,还可以为基于表型数据的研究提供有力支持。
胡松涛 , 翟瑞芳 , 王应华 , 刘志 , 朱剑忠 , 任荷 , 杨万能 , 宋鹏 . 基于多源数据的马铃薯植株表型参数提取[J]. 智慧农业, 2023 , 5(1) : 132 -145 . DOI: 10.12133/j.smartag.SA202302009
Crops have diverse structures and complex growth environments. RGB image data can reflect the texture and color features of plants accurately, while 3D data contains information about crop volume. The combination of RGB image and 3D point cloud data can achieve the extraction of two-dimensional and three-dimensional phenotypic parameters of crops, which is of great significance for the research of phenomics methods. In this study, potatoe plants were chosen as the research subject, and RGB cameras and laser scanners were used to collect 50 potato RGB images and 3D laser point cloud data. The segmentation accuracy of four deep learning semantic segmentation methods, OCRNet, UpNet, PaNet, and DeepLab v3+, were compared and analyzed for the RGB images. OCRNet, which demonstrated higher accuracy, was used to perform semantic segmentation on top-view RGB images of potatoes. Mean shift clustering algorithm was optimized for laser point cloud data processing, and single-plant segmentation of laser point cloud data was completed. Stem and leaf segmentation of single-plant potato point cloud data were accurately performed using Euclidean clustering and K-Means clustering algorithms. In addition, a strategy was proposed to establish a one-to-one correspondence between RGB images and point clouds of single-plant potatoes using pot numbering. 8 2D phenotypic parameters and 10 3D phenotypic parameters, including maximum width, perimeter, area, plant height, volume, leaf length, and leaf width, etc., were extracted from RGB images and laser point clouds, respectively. Finally, the accuracy of three representative and easily measurable phenotypic parameters, leaf number, plant height, and maximum width were evaluated. The mean absolute percentage errors (MAPE) were 8.6%, 8.3% and 6.0%, respectively, while the root mean square errors (RMSE) were 1.371 pieces, 3.2 cm and 1.86 cm, respectively, and the determination coefficients (R2) were 0.93, 0.95 and 0.91, respectively. The research results indicated that the extracted phenotype parameters can accurately and efficiently reflect the growth status of potatoes. Combining the RGB image data of potatoes with three-dimensional laser point cloud data can fully exploit the advantages of the rich texture and color characteristics of RGB images and the volumetric information provided by three-dimensional point clouds, achieving non-destructive, efficient, and high-precision extraction of two-dimensional and three-dimensional phenotype parameters of potato plants. The achievements of this study could not only provide important technical support for the cultivation and breeding of potatoes but also provide strong support for phenotype-based research.
1 | 翁杨, 曾睿, 吴陈铭, 等. 基于深度学习的农业植物表型研究综述[J]. 中国科学(生命科学), 2019, 49(6): 698-716. |
WENG Y, ZENG R, WU C M, et al. A survey on deep-learning-based plant phenotype research in agriculture[J]. Scientia sinica (vitae), 2019, 49(6): 698-716. | |
2 | GERMAIN C, ROUSSEAUD R, GRENIER G. Non destructive counting of wheatear with picture analysis[C]//Fifth International Conference on Image Processing and its Applications. London, UK: IET, 2002: 435-439. |
3 | 王传宇, 郭新宇, 杜建军, 等. 田间环境下玉米叶片图像分割方法及装置: CN106296662B[P]. 2019-07-02. |
WANG C Y, GUO X Y, DU J J, et al. Corn leaf image segmentation method and apparatus in field environment: CN106296662B[P]. 2019-07-02. | |
4 | LORESCO P J M, VALENZUELA I C, DADIOS E P. Color space analysis using KNN for lettuce crop stages identification in smart farm setup[C]// TENCON 2018—2018 IEEE Region 10 Conference. Piscataway, NJ, USA: IEEE, 2019: 2040-2044. |
5 | GRINBLAT G L, UZAL L C, LARESE M G, et al. Deep learning for plant identification using vein morphological patterns[J]. Computers and electronics in agriculture, 2016, 127: 418-424. |
6 | LEE S H, CHAN C S, WILKIN P, et al. Deep-plant: Plant identification with convolutional neural networks[C]//2015 IEEE International Conference on Image Processing (ICIP). Piscataway, NJ, USA: IEEE, 2015: 452-456. |
7 | FAN Z Q, SUN N, QIU Q, et al. A high-throughput phenotyping robot for measuring stalk diameters of maize crops[C]// 2021 IEEE 11th Annual International Conference on CYBER Technology in Automation, Control, and Intelligent Systems (CYBER). Piscataway, NJ, USA: IEEE, 2021: 128-133. |
8 | 王晓婷, 赵展, 王阳, 等. 基于改进Mask R-CNN的植物表型智能检测算法[J]. 中国农机化学报, 2022, 43(8): 151-157. |
WANG X T, ZHAO Z, WANG Y, et al. Intelligent detection algorithm of plant phenotype based on improved Mask R: CNN[J]. Journal of Chinese agricultural mechanization, 2022, 43(8): 151-157. | |
9 | ZHANG J Y, ZHAO X L, CHEN Z, et al. A review of deep learning-based semantic segmentation for point cloud[J]. IEEE access, 2019, 7: 179118-179133. |
10 | 周静静, 郭新宇, 吴升, 等. 基于多视角图像的植物三维重建研究进展[J]. 中国农业科技导报, 2019, 21(2): 9-18. |
ZHOU J J, GUO X Y, WU S, et al. Research progress on plant three-dimensional reconstruction based on multi-view images[J]. Journal of agricultural science and technology, 2019, 21(2): 9-18. | |
11 | WANG Y J, WEN W L, WU S, et al. Maize plant phenotyping: Comparing 3D laser scanning, multi-view stereo reconstruction, and 3D digitizing estimates[J]. Remote sensing, 2018, 11(1): ID 63. |
12 | WANG Y W, CHEN Y F. Non-destructive measurement of three-dimensional plants based on point cloud[J]. Plants (basel, Switzerland), 2020, 9(5): 571. |
13 | 阳旭, 胡松涛, 王应华, 等. 利用多时序激光点云数据提取棉花表型参数方法[J]. 智慧农业(中英文), 2021, 3(1): 51-62. |
YANG X, HU S T, WANG Y H, et al. Cotton phenotypic trait extraction using multi-temporal laser point clouds[J]. Smart agriculture, 2021, 3(1): 51-62. | |
14 | 屈冬玉, 谢开云, 金黎平, 等. 中国马铃薯产业发展与食物安全[J]. 中国农业科学, 2005, 38(2): 358-362. |
QU D Y, XIE K Y, JIN L P, et al. Development of potato industry and food security in China[J]. Scientia agricultura Sinica, 2005, 38(2): 358-362. | |
15 | 张兆国, 张振东, 李加念, 等. 采用改进YOLOv4模型检测复杂环境下马铃薯[J]. 农业工程学报, 2021, 37(22): 170-178. |
ZHANG Z G, ZHANG Z D, LI J N, et al. Potato detection in complex environment based on improved YOLOv4 model[J]. Transactions of the Chinese society of agricultural engineering, 2021, 37(22): 170-178. | |
16 | 赵越, 赵辉, 姜永成, 等. 基于深度学习的马铃薯叶片病害检测方法[J]. 中国农机化学报, 2022, 43(10): 183-189. |
ZHAO Y, ZHAO H, JIANG Y C, et al. Detection method of potato leaf diseases based on deep learning[J]. Journal of Chinese agricultural mechanization, 2022, 43(10): 183-189. | |
17 | 谭彧, 田芳, 刘星星, 等. 基于RGB-D相机的马铃薯图像采集装置及芽眼识别和定位方法: CN108830272A[P]. 2018-11-16. |
TAN Y, TIAN F, LIU X X, et al. Potato image acquisition device based on RGB-D cameras and method for identifying and locating bud eye: CN108830272A[P]. 2018-11-16. | |
18 | 冯慧, 杨万能, 叶军立, 等. 一种用于盆栽水稻高光谱自动成像的双通道自动输送装置及控制方法: CN107720213B[P]. 2020-10-27. |
FENG H, YANG W N, YE J L, et al. Automatic two-channel conveying device for automatic hyperspectral imaging of rice plants and control method: CN107720213B[P]. 2020-10-27. | |
19 | YUAN Y H, CHEN X K, CHEN X L, et al. Segmentation transformer: Object-contextual representations for semantic segmentation[J/OL]. arXiv: , 2019. |
20 | 宋立鹏. 室外场景三维点云数据的分割与分类[D]. 大连: 大连理工大学, 2016. |
SONG L P. Segmentation and classification of 3D point cloud data in outdoor scenes[D]. Dalian: Dalian University of Technology, 2016. | |
21 | RUSU R B. Semantic 3D object maps for everyday manipulation in human living environments[J]. KI-Künstliche Intelligenz, 2010, 24: 345-348. |
22 | HARTIGAN J A, WONG M A. A K-means clustering algorithm[J]. Journal of the royal statistical society: Series C (applied statistics), 1979, 28(1): 100-108. |
23 | 方敏, 徐俊艳, 王建平, 等. 一种新的文本图像二值化方法[J]. 合肥工业大学学报(自然科学版), 2001, 24(2): 166-169 |
FANG M, XU J Y, WANG J P, et al. A new binarization algorithm for document image[J]. Journal of Hefei university of technology (Natural Science), 2001, 24(2): 166-169 | |
24 | LECUN Y, CORTES C, BURGES C J C. THE MNIST DATABASE of handwritten digits [EB/OL]. (1996-11-08) [2023-01-10]. |
25 | KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Commun ACM, 2017, 60(6): 84-90. |
26 | 熊雄. 基于深度学习的大田水稻稻穗分割及无损产量预估研究[D]. 武汉: 华中科技大学, 2018. |
XIONG X. Study on rice ear segmentation and lossless yield prediction of field rice based on deep learning[D].Wuhan: Huazhong University of Science and Technology, 2018. | |
27 | 杨万里, 段凌凤, 杨万能. 基于深度学习的水稻表型特征提取和穗质量预测研究[J]. 华中农业大学学报, 2021, 40(1): 227-235. |
YANG W L, DUAN L F, YANG W N. Deep learning-based extraction of rice phenotypic characteristics and prediction of rice panicle weight[J]. Journal of Huazhong agricultural university, 2021, 40(1): 227-235. | |
28 | CHEN L C, ZHU Y K, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]// European conference on computer vision. Berlin, German: Springer, 2018: 833-851. |
29 | XIAO T T, LIU Y C, ZHOU B L, et al. Unified perceptual parsing for scene understanding[C]// European Conference on Computer Vision. Berlin, German: Springer, 2018: 432-448. |
30 | LIU S, QI L, QIN H F, et al. Path aggregation network for instance segmentation[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, USA: IEEE, 2018: 8759-8768. |
/
〈 |
|
〉 |