Abstract:

[Objective] There's a growing demand among plant cultivators and breeders for efficient methods to acquire plant phenotypic traits at high throughput, facilitating the establishment of mappings from phenotypes to genotypes. By integrating mobile phenotyping platforms with improved instance segmentation techniques, researchers have achieved a significant advancement in the automation and accuracy of phenotypic data extraction. Addressing the need for rapid extraction of leaf age and canopy width phenotypes in strawberry plants cultivated in controlled environments, this study introduces a novel high-throughput phenotyping extraction approach leveraging a mobile phenotyping platform and instance segmentation technology. [Methods] Data acquisition was conducted using a compact mobile phenotyping platform equipped with an array of sensors, including an RGB sensor, and edge control computers, capable of capturing overhead images of potted strawberry plants in greenhouses. Targeted adjustments to the network structure were made to develop an enhanced convolutional neural network (Mask R-CNN) model for processing strawberry plant image data and rapidly extracting plant phenotypic information. The model initially employed a split-attention networks (ResNeSt) backbone with a group attention module, replacing the original network to improve the precision and efficiency of image feature extraction. During training, the model adopted the Mosaic method, suitable for instance segmentation data augmentation, to expand the dataset of strawberry images. Additionally, it optimized the original cross-entropy classification loss function with a binary cross-entropy loss function to achieve better detection accuracy of plants and leaves. Based on this, the improved Mask R-CNN description involves post-processing of training results. It utilized the positional relationship between leaf and plant masks to statistically count the number of leaves. Additionally, it employed segmentation masks and image calibration against true values to calculate the canopy width of the plant. [Results and Discussions] This research conducted a thorough evaluation and comparison of the performance of an improved Mask R-CNN model, underpinned by the ResNeSt-101 backbone network. This model achieved a commendable mask accuracy of 80.1% and a detection box accuracy of 89.6%. It demonstrated the ability to efficiently estimate the age of strawberry leaves, demonstrating a high plant detection rate of 99.3% and a leaf count accuracy of 98.0%. This accuracy marked a significant improvement over the original Mask R-CNN model and meeting the precise needs for phenotypic data extraction. The method displayed notable accuracy in measuring the canopy widths of strawberry plants, with errors falling below 5% in about 98.1% of cases, highlighting its effectiveness in phenotypic dimension evaluation. Moreover, the model operated at a speed of 12.9 frames per second (FPS) on edge devices, effectively balancing accuracy and operational efficiency. This speed proved adequate for real-time applications, enabling rapid phenotypic data extraction even on devices with limited computational capabilitie. [Conclusions] This study successfully deployed a mobile phenotyping platform combined with instance segmentation techniques to analyze image data and extract various phenotypic indicators of strawberry plant. Notably, the method demonstrates remarkable robustness. The seamless fusion of mobile platforms and advanced image processing methods not only enhances efficiency but also ignifies a shift towards data-driven decision-making in agriculture.

Key words: mobile phenotype platform, instance segmentation, strawberry plant phenotype, leaf age, plant crown width, Mask R-CNN, ResNeSt