Tassel Segmentation of Maize Point Cloud Based on Super Voxels Clustering and Local Features

doi:10.12133/j.smartag.2021.3.1.202102-SA001

Abstract

Abstract:

Accurate and high-throughput maize plant phenotyping is vital for crop breeding and cultivation research. Tassel-related phenotypic parameters are important agronomic traits. However, fully automatic and fine tassel organ segmentation of maize shoots from three-dimensional (3D) point clouds is still challenging. To address this issue, a tassel point cloud segmentation method based on point cloud super voxels clustering and local geometric features was proposed in this study. Firstly, the undirected graph of the maize plant point cloud was established, the edge weights were calculated by using the difference of normal vectors, and the spectral clustering method was used to cluster the point cloud to form multiple super voxel sub-regions. Then, the principal component analysis method was used to find the two end regions of the plant and based on the observation of the straight direction of the bottom stem regions, the top and bottom regions were distinguished by the point cloud linear features. Finally, the tassel points were identified based on the plane local features of the point cloud. The sub-regions of the top region of the plant were classified into leaf regions, tassel regions, and mixed regions by plane local features of the point cloud, the tassel points in the tassel sub-region, and the mixed region were the finally segmented tassel point clouds. In this study, 15 mature maize plants with 3 point cloud densities were tested. Compared with the ground truth segmented manually, the average F1 scores of the tassel segmentation were 0.763, 0.875 and 0.889 when the point cloud density was 0.8/cm, 1.3/cm, and 1.9/cm, respectively. The segmentation accuracy of this method increased with the increase of plant point cloud density. The increase of point cloud density and the number of point clouds mainly affected the calculation results of point cloud plane features in tassel segmentation. When the number of point clouds was small, the top leaf point cloud was relatively sparse. Therefore, the difference between the plane feature of the leaf point and the plane feature of the tassel point was not obvious, which led to the increase of the misclassification of the point cloud. However, the time complexity of the algorithm was O(n³), so the increase in the density and number of point clouds would lead to a significant increase in the running time. Considering the segmentation accuracy and running time, the research obtained the best effect on the mature maize plants with a point cloud density of 1.3/cm and an average number of 15,000. The segmentation F1 score reached 0.875 and the running time was 6.85 s. The results showed that this method could extract tassels from maize plant point cloud, and provided technical support for the research and application of high-throughput phenotyping and three-dimensional reconstruction of maize.

Key words: maize tassel, 3D point cloud segmentation, phenotyping, super voxels clustering, local features, principal component analysis

CLC Number:

ZHU Chao, WU Fan, LIU Changbin, ZHAO Jianxiang, LIN Lili, TIAN Xueying, MIAO Teng. Tassel Segmentation of Maize Point Cloud Based on Super Voxels Clustering and Local Features[J]. Smart Agriculture, 2021, 3(1): 75-85.

References

1	TESTER M, LANGRIDGE P. Breeding technologies to increase crop production in a changing world[J]. Science, 2010, 327: 818-822.
2	DUNCAN W G, WILLIAMS W A, LOOMIS R S. Tassels and the productivity of maize[J]. Crop Science, 1967, 7: 7-9.
3	MEGHJI M R, DUDLEY J W, LAMBERT R J, et al. Inbreeding depression, inbred and hybrid grain yields, and other traits of maize genotypes representing three eras[J]. Crop Science, 1984, 24(3): 545-549.
4	BROWN P J, UPADYAYULA N, MAHONE G S, et al. Distinct genetic architectures for male and female inflorescence traits of maize[J]. PLoS Genet, 2011,7(11): ID e1002383.
5	ZHANG Y, ZHANG N. Imaging technologies for plant high-throughput phenotyping: A review[J]. Frontiers of Agricultural Science and Engineering, 2018, 5(4): 406-419.
6	FERNANDA D M, GEMMA M, CAROLINA R A, et al. Yielding to the image: How phenotyping reproductive growth can assist crop improvement and production[J]. Plant Science, 2018, 282: 73-82.
7	GAGE J L, MILLER N D, SPALDING E P, et al. TIPS: A system for automated image-based phenotyping of maize tassels[J]. Plant Methods, 2017, 13(1): 27-32.
8	YE M, CAO Z, YU Z. An image-based approach for automatic detecting tasseling stage of maize using spatio-temporal saliency[C]// SPIE Conference on Multispectral Image Processing and Pattern Recognition. Washington, USA: SPIE Digital library, 2013.
9	KURTULMU F, KAVDR S. Detecting corn tassels using computer vision and support vector machines[J]. Expert Systems with Applications, 2014, 41(16): 7390-7397.
10	LU H, CAO Z, XIAO Y, et al. Fine-grained maize tassel trait characterization with multi-view representations[J]. Computers and Electronics in Agriculture, 2015, 118: 143-158.
11	LU H, CAO Z, XIAO Y, et al. TasselNet: Counting maize tassels in the wild via local counts regression network[J]. Plant Methods, 2017, 13(1):1-17.
12	MAKANZA R, ZAMAN-ALLAH M, CAIRNS J E, et al. High-throughput method for ear phenotyping and kernel weight estimation in maize using ear digital imaging[J]. Plant Methods, 2018, 15(1): ID 49.
13	王传宇, 郭新宇, 吴升, 等. 基于计算机视觉的玉米果穗三维重建方法[J]. 农业机械学报, 2014, 45(9): 274-279.
13	WANG C, GUO X, WU S, et al. Three dimensional reconstruction of maize ear based on computer vision[J]. Transactions of the CSAM, 2014, 45(9): 274-279.
14	温维亮, 郭新宇, 杨涛, 等. 玉米果穗点云分割方法研究[J]. 系统仿真学报, 2017, 29(12): 3030-3034, 3041.
14	WEN W, GUO X, YANG T, et al. Point cloud segmentation method of maize ear[J]. Journal of System Simulation, 2017, 29(12): 3030-3034, 3041.
15	温维亮, 王勇健, 许童羽, 等. 基于三维点云的玉米果穗几何建模[J]. 中国农业科技导报, 2016, 18(5): 88-93.
15	WEN W, WANG Y, XU T, et al. Geometric modeling of maize ear based on three-dimensional point cloud[J]. Journal of Agricultural Science and Technology, 2016, 18(5): 88-93.
16	YU Z, ZHOU H, LI C. An image based automatic recognition method for the flowering stage of maize[C]// International Symposium on Multispectral Image Processing and Pattern Recognition. Washington, USA: SPIE Digital library, 2019.
17	BRICHET N, FOURNIER C, TURC O, et al. A robot-assisted imaging pipeline for tracking the growths of maize ear and silks in a high-throughput phenotyping platform[J]. Plant Methods, 2017, 13(1): ID 96.
18	JIA H, QU M H, WANG G, et al. Dough-stage maize (Zea mays L.) ear recognition based on multiscale hierarchical features and multifeature fusion[J]. Mathematical Problems in Engineering, 2020, 2020(2): ID 5582598.
19	PAULUS S, SCHUMANN H, KUHLMANN H, et al. High-precision laser scanning system for capturing 3D plant architecture and analysing growth of cereal plants[J]. Biosystems Engineering, 2014, 121: 1-11.
20	PAULUS S, DUPUIS, J, RIEDEL, S, et al. Automated analysis of barley organs using 3D laser scanning: An approach for high throughput phenotyping[J]. Sensors, 2014, 14(7): 12670-12686.
21	CHéNé Y, ROUSSEAU D, LUCIDARME P, et al. On the use of depth camera for 3D phenotyping of entire plants[J]. Computers and Electronics in Agriculture, 2012, 82: 122-127.
22	BUSEMEYER L, MENTRUP D, M?LLER K, et al. BreedVision－A multi-sensor platform for non-destructive field-based phenotyping in plant breeding[J]. Sensors, 2013, 13(3): 2830-2847.
23	LIN Y. LiDAR: An important tool for next-generation phenotyping technology of high potential for plant phenomics?[J]. Computers and Electronics in Agriculture, 2015, 119: 61-73.
24	GARRIDO M, PARAFOROS D S, REISER D, et al. 3D maize plant reconstruction based on georeferenced overlapping lidar point clouds[J]. Remote Sensing, 2015, 7(12): 17077-17096.
25	韩东, 杨贵军, 杨浩, 等. 基于立体视觉的玉米雄穗三维信息提取[J]. 农业工程学报, 2018, 34(11): 166-173.
25	HAN D, YANG G, YANG H, et al. Three dimensional information extraction from maize tassel based on stereoscopic vision[J]. Transactions of the CSAE, 2018, 34(11): 166-173.
26	MILLER F P, VANDOME A F, MCBREWSTER J. KD-TREE[M]. San Francisco, California, USA: Alpha Press, 2009.
27	RUSU R B, COUSINS S. 3D is here: Point cloud library (PCL)[C]// IEEE International Conference on Robotics & Automation. New Jersey, USA: IEEE Press, 2011.
28	GROTH D, HARTMANN S, KLIE S, et al. Principal components analysis[J]. Methods in Molecular Biology, 2013, 930: 527-547.
29	XIANG L, BAO Y, TANG L, et al. Automated morphological traits extraction for sorghum plants via 3D point cloud data analysis[J]. Computers and Electronics in Agriculture, 2019, 162: 951-961.
30	仇瑞承, 张漫, 魏爽, 等. 基于 RGB-D相机的玉米茎粗测量方法[J]. 农业工程学报, 2017, 33(S1): 170-176.
30	QIU R, ZHANG M, WEI S, et al. Method for measurement of maize stem diameters based on RGB-D camera[J]. Transactions of the CSAE, 2017, 33(S1): 170-176.
31	JIN S, SU Y, WU F, et al. Stem-leaf segmentation and phenotypic trait extraction of individual maize using terrestrial LiDAR data[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(3): 1336-1346.
32	WU S, WEN W, WANG Y, et al. MVS-Pheno: A portable and low-cost phenotyping platform for maize shoots using multiview stereo 3D reconstruction[J]. Plant Phenomics. 2020, 2020(2): ID 1848437.

[1]	HU Songtao, ZHAI Ruifang, WANG Yinghua, LIU Zhi, ZHU Jianzhong, REN He, YANG Wanneng, SONG Peng. Extraction of Potato Plant Phenotypic Parameters Based on Multi-Source Data [J]. Smart Agriculture, 2023, 5(1): 132-145.
[2]	BAI Geng, GE Yufeng. Crop Stress Sensing and Plant Phenotyping Systems: A Review [J]. Smart Agriculture, 2023, 5(1): 66-81.
[3]	GAO Zhen, ZHAO Chunjiang, YANG Guiyan, DONG Daming. Typical Raman Spectroscopy Ttechnology and Research Progress in Agriculture Detection [J]. Smart Agriculture, 2022, 4(2): 121-134.
[4]	YANG Xu, HU Songtao, WANG Yinghua, YANG Wanneng, ZHAI Ruifang. Cotton Phenotypic Trait Extraction Using Multi-Temporal Laser Point Clouds [J]. Smart Agriculture, 2021, 3(1): 51-62.
[5]	Fu Yuanyuan, Yang Guijun, Duan Dandan, Zhang Yongtao, Gu Xiaohe, Yang Xiaodong, Xu Xingang, Li Zhenhai. Comparison analysis of spatial and spectral feature in vegetation classification based on AVIRIS hyperspectral image [J]. Smart Agriculture, 2020, 2(1): 68-76.
[6]	Liu Shouyang, Jin Shichao, Guo Qinghua, Zhu Yan, Baret Fred. An algorithm for estimating field wheat canopy light interception based on Digital Plant Phenotyping Platform [J]. Smart Agriculture, 2020, 2(1): 87-98.
[7]	Xu Lingxiang, Chen Jiawei, Ding Guohui, Lu Wei, Ding Yanfeng, Zhu Yan, Zhou Ji. Indoor phenotyping platforms and associated trait measurement: Progress and prospects [J]. Smart Agriculture, 2020, 2(1): 23-42.