Abstract:

[Objective] The proliferation of kiwifruit trees severely overlaps, resulting in a complex canopy structure, rendering it impossible to extract their skeletons or predict their canopies using conventional methods. The objective of this research is to propose a crown segmentation method that integrates skeleton information by optimizing image processing algorithms and developing a new scheme for fusing winter and summer information. In cases where fruit trees are densely distributed, achieving accurate segmentation of fruit tree canopies in orchard drone images can efficiently and cost-effectively obtain canopy information, providing a foundation for determining summer kiwifruit growth size, spatial distribution, and other data. Furthermore, it facilitates the automation and intelligent development of orchard management. [Methods] The 4- to 8-year-old kiwifruit trees were chosen and remote sensing images of winter and summer via unmanned aerial vehicles were obtain as the primary analysis visuals. To tackle the challenge of branch extraction in winter remote sensing images, a convolutional attention mechanism was integrated into the PSP-Net network, along with a joint attention loss function. This was designed to boost the network's focus on branches, enhance the recognition and targeting capabilities of the target area, and ultimately improve the accuracy of semantic segmentation for fruit tree branches.For the generation of the skeleton, digital image processing technology was employed for screening. The discrete information of tree branches was transformed into the skeleton data of a single fruit tree using growth seed points. Subsequently, the semantic segmentation results were optimized through mathematical morphology calculations, enabling smooth connection of the branches. In response to the issue of single tree canopy segmentation in summer, the growth characteristics of kiwifruit trees were taken into account, utilizing the outward expansion of branches growing from the trunk.The growth of tree branches was simulated by using morphological expansion to predict the summer canopy. The canopy prediction results were analyzed under different operators and parameters, and the appropriate expansion operators along with their corresponding operation lengths were selected. The skeleton of a single tree was extracted from summer images. By combining deep learning with mathematical morphology methods through the above steps, the optimized single tree skeleton was used as a prior condition to achieve canopy segmentation. [Results and Discussions] In comparison to traditional methods, the accuracy of extracting kiwifruit tree canopy information images at each stage of the process has been significantly enhanced. The enhanced PSP Net was evaluated using three primary regression metrics: pixel accuracy (PA), mean intersection over union ratio (MIoU), and weighted F₁ Score (WF₁). The PA, MIoU and WF₁ of the improved PSP-Net were 95.84%, 95.76% and 95.69% respectively, which were increased by 12.30%, 22.22% and 17.96% compared with U-Net, and 21.39% , 21.51% and 18.12% compared with traditional PSP-Net, respectively. By implementing this approach, the skeleton extraction function for a single fruit tree was realized, with the predicted PA of the canopy surpassing 95%, an MIoU value of 95.76%, and a WF₁ of canopy segmentation approximately at 94.07%.The average segmentation precision of the approach surpassed 95%, noticeably surpassing the original skeleton's 81.5%. The average conformity between the predicted skeleton and the actual summer skeleton stand at 87%, showcasing the method's strong prediction performance. Compared with the original skeleton, the PA, MIoU and WF₁ of the optimized skeleton increased by 13.2%, 10.9% and 18.4%, respectively. The continuity of the predicted skeleton had been optimized, resulting in a significant improvement of the canopy segmentation index. The solution effectively addresses the issue of semantic segmentation fracture, and a single tree canopy segmentation scheme that incorporates skeleton information could effectively tackle the problem of single fruit tree canopy segmentation in complex field environments. This provided a novel technical solution for efficient and low-cost orchard fine management. [Conclusions] A method for extracting individual kiwifruit plant skeletons and predicting canopies based on skeleton information was proposed. This demonstrates the enormous potential of drone remote sensing images for fine orchard management from the perspectives of method innovation, data collection, and problem solving. Compared with manual statistics, the overall efficiency and accuracy of kiwifruit skeleton extraction and crown prediction have significantly improved, effectively solving the problem of case segmentation in the crown segmentation process.The issue of semantic segmentation fragmentation has been effectively addressed, resulting in the development of a single tree canopy segmentation method that incorporates skeleton information. This approach can effectively tackle the challenges of single fruit tree canopy segmentation in complex field environments, thereby offering a novel technical solution for efficient and cost-effective orchard fine management. While the research is primarily centered on kiwifruit trees, the methodology possesses strong universality. With appropriate modifications, it can be utilized to monitor canopy changes in other fruit trees, thereby showcasing vast application potential.

Key words: extraction of fruit tree skeleton, canopy prediction, deep learning, machine vision, digital image processing, drone remote sensing, crown prediction