Abstract:

[Objective] To address the challenges posed by the concentrated flowering period of kiwifruit, the relatively low efficiency and inconsistent quality of manual supplementary pollination, and the frequent approach and retraction motions within dense flower distributions in trellis canopies during flower-by-flower robotic operations, a method integrating visual localization of kiwifruit flower centers, three-dimensional (3D) target generation, and intra-cluster sweeping was proposed for continuous pollination operations. The visual perception performance of the proposed method and feasibility of robotic operation were evaluated. [Methods] An RGB-D camera was used to synchronously acquire color images and aligned depth images of the trellis canopy, and a kiwifruit flower detection and flower-center keypoint dataset containing 1 001 images and 11 794 flower targets was constructed. The flower center was defined as the geometric center of the central stigma cluster of a female flower in its two-dimensional image projection. Three visibility levels were defined according to whether the flower-center was clearly visible, occluded but still locatable, and unable to be reliably annotated. With YOLO26n-Pose as the baseline, the original pose-estimation keypoint detection head was replaced by a flower-center offset prediction head that estimates the flower-center position relative to the center of the detected flower bounding box. Center auxiliary supervision, anisotropic Gaussian probabilistic flower-center localization, and a normalized Wasserstein distance (NWD) bounding-box loss were introduced, while the object keypoint similarity (OKS)-based keypoint training loss was excluded from the final model. After the model output flower bounding boxes, flower-center coordinates, and confidence scores, invalid depth values were removed within a local depth window around each flower-center, and the median of the valid depth values was used as the target depth. A depth reliability score was further calculated based on the valid-depth ratio, local depth standard deviation, and center-depth consistency. Flower-center pixels satisfying the quality criteria were back-projected using the camera intrinsic parameters and transformed using the hand-eye calibration extrinsic parameters to generate 3D operation targets in the robot base coordinate system. The system combined cross-frame target identification, target-quality filtering, and state management to prevent completed targets from being repeatedly issued. Spatially adjacent targets were then clustered and ordered using a nearest-neighbor strategy, and centripetal Catmull-Rom splines were employed to generate continuous intra-cluster sweeping paths. The proposed method was evaluated through visual-model ablation and comparison experiments, comparative indoor experiments using artificial flower clusters, and system experiments conducted at four orchard operating positions. [Results and Discussions] The final model achieved bounding-box mean average precision (mAP) values of 84.82% for mAP₅₀ and 54.00% for mAP_50:95, respectively, and keypoint mAP₅₀ and mAP_50:95 values of 88.13% and 87.78%, respectively. The miss rate (Miss@0.5) was 22.94%, and the mean pixel error of flower-center localization for successfully matched targets was 2.56 px. Compared with the YOLO26n-Pose baseline, Miss@0.5 was reduced by 6.71 percentage points, while the mean flower-center localization error remained at 2.56 px. In the comparative indoor experiments using artificial flower clusters, each operation mode included a total of 135 targets. Compared with flower-by-flower operation, the intra-cluster sweeping mode reduced the total robotic end-effector path length from 36.68 m to 19.74 m, and decreased the average operation time per flower from 9.68 s to 5.52 s. In the orchard experiments, 92 targets were distributed across four operating positions. The motion execution success rate, target operation completion rate, and average operation time per flower were 95.80%, 84.78%, and 6.82 s, respectively. Conclusions In the typical orchard scenarios evaluated in this study, the proposed method achieved a continuous closed-loop operation workflow encompassing flower detection, flower-center localization, 3D operation target generation, target scheduling, and robotic manipulator execution. Under the tested indoor conditions with fixed artificial flower clusters, the intra-cluster sweeping strategy reduced the end-effector path length and average operation time per flower while maintaining or slightly improving the target operation completion rate. These results provide a technical basis for visual perception and operation planning in intelligent kiwifruit pollination equipment.

Key words: kiwifruit, flower-center visual localization, RGB-D vision, sweeping operation, robotic pollination