关键词: 结球甘蓝, 语义分割, 叶球识别, 注意力机制, 特征选择, 特征对齐

Abstract:

Objective Kale is an important bulk vegetable crop in the world, and its main growth characteristics are outer leaves and leaf bulbs, while the leaf bulb traits of kale are crucial for the adjustment of water and fertilizer parameters in the field and the final yield. However, in practical field conditions, factors such as soil quality, light exposure, leaf overlap, and shading can affect kale growth. The similarity in color and texture between the leaf bulbs and outer leaves complicates the segmentation process for existing recognition models. In this paper, the segmentation of kale outer leaves and leaf bulbs was proposed under the complex background in the field, using pixel values to determine leaf bulb size for intelligent field management. A semantic segmentation algorithm UperNet-ESA was proposed to be able to quickly and accurately segment the outer leaf and leaf bulb of nodular kale in field scenes using the morphological features of the leaf bulb and outer leaf of nodular kale, in order to realize the intelligent management of nodular kale in the field. Methods In this research, the UperNet-ESA semantic segmentation algorithm, which used the unified perceptual parsing network (UperNet) as an efficient semantic segmentation framework, was more suitable for extracting features of crops in complex environments as it could integrate semantic information from different scales. The backbone network was improved using ConvNeXt, responsible for feature extraction in the model. Due to the challenges posed by the similarity between kale leaf bulbs and outer leaves, as well as leaf overlap hindering accurate target contour localization, the baseline network struggled with low accuracy. ConvNeXt effectively combines the strengths of convolutional neural networks (CNN) and Transformers, building upon ResNet50 and incorporating design principles from Swin Transformer, resulting in a highly effective network structure. The simplicity of the ConvNeXt design enhances segmentation accuracy with minimal model complexity, making it a top-performing CNN. In this study, the ConvNeXt-B version was chosen based on the computational complexity and background characteristics of the knotweed kale image dataset. To enhance the model's perceptual acuity, the block ratios for each stage were set at 3:3:27:3 and the channel numbers at 128, 256, 512 and 1 024, respectively. Given the visual similarity between kale leaf bulbs and outer leaves, a high-efficiency channel attention mechanism was integrated into the backbone network to improve feature extraction in the leaf bulb region. By incorporating attention weights into feature mapping via residual inversion, attention parameters were cyclically trained within each block. This iterative process generated feature maps with attentional weights, facilitating the repeated training of attentional parameters and enhancing the capture of global feature information. Addressing issues arising from direct pixel addition between up-sampling and local features, which led to misaligned context in feature maps and potential erroneous classifications at kale leaf boundaries, a feature alignment module and feature selection module were introduced into the feature pyramid network to refine target boundary information extraction and enhance model segmentation. Results and Discussions The UperNet-ESA semantic segmentation model outperforms the current mainstream UNet model, PSPNet model, DeepLabV3+ model in terms of segmentation accuracy, where mIoU and mPAreached 92.45% and 94.32%, respectively, and the inference speed could reach up to 16.6 fps. MPA was better than that of the UNet model, the PSPNet model, the ResNet-50 based, MobilenetV2, and DeepLabV3+ model with Xception as the backbone. mPA values were improved by 11.52%, 13.56%, 8.68%, 4.31%, and 6.21%, respectively. Similarly, the mIoU were 12.21%, 13.04%, 10.65%, 3.26% and 7.11% higher than that of the UNet-based model, the PSPNet model, and the DeepLabV3+ model based on the ResNet-50, MobilenetV2, and Xception backbones, respectively. The main reason was that the introduction of the ECA module and the improvement of the feature pyramid network in this model strengthen the judgement of the target features at each stage to obtain effective global contextual information. In addition, although the PSPNet model had the fastest inference speed, the overall accuracy was too low to be suitable for constructing kale semantic segmentation models. The inference speed of this model was faster than that of all other network models. In the comprehensive analysis, this model could spend as little inference time as possible under the premise of guaranteeing the accuracy of the model in order to achieve a balance between the recognition accuracy and the recognition speed of kale leaf bulb, so as to provide a theoretical basis for intelligent field management. The UperNet-ESA semantic segmentation model proposed demonstrates superior performance compared to the current mainstream UNet model, PSPNet model, and DeepLabV3+ model in terms of segmentation accuracy. It achieved mIoU and mPA scores of 92.45% and 94.32% respectively, with an impressive inference speed of up to 16.6 f/s. The mPA outperformed those of the UNet model, PSPNet model, ResNet-50 based model, MobilenetV2, and DeepLabV3+ model with Xception as the backbone, showing improvements of 11.52%, 13.56%, 8.68%, 4.31%, and 6.21% respectively. Similarly, the mIoU scores were notably higher by 12.21%, 13.04%, 10.65%, 3.26%, and 7.11% compared to the mIoU scores of the UNet-based model, PSPNet model, DeepLabV3+ model based on the ResNet-50, MobilenetV2, and Xception backbones. This performance enhancement can be attributed to the incorporation of the ECA module and enhancements to the feature pyramid network in this model, which bolster the identification of target features at each stage, enabling the extraction of effective global contextual information. Despite the PSPNet model's faster inference speed, its overall accuracy was insufficient for constructing precise kale semantic segmentation models. In contrast, this model exhibited superior inference speed compared to all other network models. Conclusions The experimental results show that the UperNet-ESA semantic segmentation model proposed in this study outperforms the original network in terms of performance; the improved model in this paper achieves the best accuracy-speed balance compared to the current mainstream semantic segmentation networks.Through comprehensive analysis, this model could lay a foundation for intelligent field management. In the upcoming research, the current model will be further optimized and enhanced, while the kale dataset will be expanded to encompass a greater variety of samples of nodulated kale leaf bulbs. This expansion aims to offer more robust and comprehensive theoretical underpinning for intelligent kale field management.