Hand-Eye Configuration and Chassis Stepping Optimization for a Dual-Arm Grape-Picking Robot

doi:10.12133/j.smartag.SA202606010

Abstract

Abstract:

[Objective] During field harvesting operations, dual-arm robots often face challenges such as insufficient workspace coverage of the two arms, high overlap in collaborative working areas, and fixed-step movement patterns that struggle to adapt to spatial variations in cluster distribution. These issues lead to excessive workspace overlap between the two manipulators, low workspace utilization, and missed harvests due to rigid step sizes, thereby limiting the robot's continuous operation capability and overall harvesting efficiency. To address these problems, the optimization of hand-eye configuration and an adaptive chassis movement strategy for dual-arm harvesting robots are investigated in this study, aiming to improve workspace utilization and enhance both continuous operation performance and overall harvesting efficiency. [Methods] A dual-arm grape harvesting robot equipped with an onboard storage-buffering system was designed. The robot consisted of a mobile chassis, a dual-arm harvesting system, a vision perception system, a storage-buffering system, and a control system. Based on operational requirements of the dual-arm collaborative harvesting system, an effective workspace model was established by intersecting the reachable domains of the robotic arms, the depth camera's field of view, and the grape cluster distribution zones. A hand-eye layout optimization model was developed considering occlusion relationships, using camera row spacing and distance from the camera to the trellis wall as optimization variables. Optimal layout parameters were determined through a two-parameter uniform grid search. Sensitivity analysis was conducted on workspace utilization and workspace overlap under different layout configurations, revealing how layout parameters affected workspace coverage and dual-arm harvesting performance, thereby enabling quantitative optimization rather than empirical parameter selection. To address difficulties in detecting small peduncles and acquiring accurate depth information under long-range "eye-to-hand" visual configurations, the vision system combined YOLOv8 object detection with geometric modeling of clusters to detect grape clusters and estimate peduncle cutting points. Target tracking was further employed to associate clusters with their respective cutting points, providing reliable target coordinates for robotic harvesting. Additionally, an adaptive chassis movement strategy based on the spatial distribution of peduncle cutting points was proposed to reduce missed harvests caused by fixed-step motion. This strategy dynamically switched between forward and compensatory steps according to whether cutting points fell within uncovered regions of the manipulator workspace, ensuring continuous coverage of the harvesting area and reducing missed clusters due to uneven distribution. Field trials were conducted across five scenarios with different cluster distributions in a standardized horizontal trellis vineyard, comparing the adaptive movement strategy against the fixed-step approach. [Results and Discussions] The optimized hand-eye layout achieved a workspace utilization rate of 89.7%, while the workspace overlap between the two manipulators was reduced to 3.5%, thereby improving coverage of the grape-cluster harvesting region. Sensitivity analysis further revealed that both camera row spacing and distance from the camera to the trellis wall significantly influenced workspace utilization and arm overlap; the optimal parameter settings effectively balanced workspace coverage and dual-arm collaboration performance. Vision experiments achieved 100% cluster detection accuracy and 96% cutting-point localization accuracy, meeting the positioning requirements of the robot. Field harvesting tests demonstrated that the robot achieved an average harvesting success rate of 94%, representing an 11% improvement over the fixed-step strategy and a reduction of approximately 12% in the missed-harvest rate. [Conclusions] The proposed robot enables stable and continuous field harvesting operations. The proposed hand-eye layout optimization method effectively enhances workspace utilization. The proposed adaptive chassis motion strategy reduces the missed-harvest rate and improves the harvesting success rate and operational stability. The research results provide a theoretical basis and engineering reference for optimizing the hand-eye system layout, designing chassis motion strategies, and achieving continuous and efficient grape harvesting in horizontal trellis systems.

Key words: robot, grape, motion planning, application test, harvesting

CLC Number:

ZHANG Chenghao, YANG Shenghui, XIA Chuandi, ZHANG Kangyang, ZHENG Yongjun, XU Liming, LI Jieyun, XUE Shaohua, LIU Xingxing. Hand-Eye Configuration and Chassis Stepping Optimization for a Dual-Arm Grape-Picking Robot[J]. Smart Agriculture, doi: 10.12133/j.smartag.SA202606010.

Figures/Tables 13

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Fig. 9

Table 1

Optimization parameters of hand-eye configuration for the picking robot

参数	数值/mm
$Y c a m$	455
$D$	937
$X w a l l$	717
$Z c a m$	135 0
$X c a m$	-220
$y w e f f L$	$- 1080,0$
$y w e f f R$	$0,1080$
$z w e f f L$	$1100,1595$
$y w b l a n k k$	$- 255,255$
$z w b l a n k k$	$[1355,1595]$

Table 1

Fig. 10

Fig. 11

Table 2

References 33

[1]	KARASTERGIOU A, GANCEL A L, JOURDES M, et al. Valorization of grape pomace: a review of phenolic composition, bioactivity, and therapeutic potential[J]. Antioxidants, 2024, 13(9): 1131.
[2]	FAO. Crops and livestock products[EB/OL]. 2023. [2026-08-24]
[3]	ZHOU X Z, ZOU X J, TANG W, et al. Unstructured road extraction and roadside fruit recognition in grape orchards based on a synchronous detection algorithm[J]. Frontiers in Plant Science, 2023, 14: 1103276.
[4]	SUN F C, LV Q R, BIAN Y C, et al. Grape target detection method in orchard environment based on improved YOLOv7[J]. Agronomy, 2025, 15(1): 42.
[5]	WANG Z H, XUN Y, WANG Y K, et al. Review of smart robots for fruit and vegetable picking in agriculture[J]. International Journal of Agricultural and Biological Engineering, 2022, 15(1): 33-54.
[6]	LI T, FENG Q C, QIU Q, et al. Occluded apple fruit detection and localization with a frustum-based point-cloud-processing approach for robotic harvesting[J]. Remote Sensing, 2022, 14(3): 482.
[7]	LEICHMAN A K. World's first tomato-picking robot set to be rolled out[EB/OL]. 2022. [2026-08-24]
[8]	TUFAIL M, IQBAL J, AHMAD R. Development of a general-purpose AI-powered robotic platform for strawberry harvesting[J]. Agriculture, 2026, 16(7): 769.
[9]	XIONG Y, GE Y Y, GRIMSTAD L, et al. An autonomous strawberry-harvesting robot: design, development, integration, and field evaluation[J]. Journal of Field Robotics, 2020, 37(2): 202-224.
[10]	SAUNDERS S. The robots that can pick kiwi-fruit[EB/OL]. 2022. [2026-08-24]
[11]	GURSOY E. Robots à deux bras pour la récolte de précision des fruits en environnement agricole[D]. Montpellier: Université de Montpellier, 2024.
[12]	JIANG Y X, LIU J Z, WANG J, et al. Development of a dual-arm rapid grape-harvesting robot for horizontal trellis cultivation[J]. Frontiers in Plant Science, 2022, 13: 881904.
[13]	LING X, ZHAO Y S, GONG L, et al. Dual-arm cooperation and implementing for robotic harvesting tomato using binocular vision[J]. Robotics and Autonomous Systems, 2019, 114(C): 134-143.
[14]	LI T, XIE F, ZHAO Z Q, et al. A multi-arm robot system for efficient apple harvesting: perception, task plan and control[J]. Computers and Electronics in Agriculture, 2023, 211: 107979.
[15]	MU L T, CUI G P, LIU Y D, et al. Design and simulation of an integrated end-effector for picking kiwifruit by robot[J]. Information Processing in Agriculture, 2020, 7(1): 58-71.
[16]	JI W T, HUANG X H, WANG S B, et al. A comprehensive review of the research of the "eye–brain–hand" harvesting system in smart agriculture[J]. Agronomy, 2023, 13(9): 2237.
[17]	HAN J W, CHO J H, KIM Y T. Design of fruit harvesting robot system based on a reachability and inverse reachability map[J]. AgriEngineering, 2025, 7(12): 417.
[18]	YIN C H, HUANG J Y, XIA Y Y, et al. Design, development, integration and field evaluation of a dual robotic arm mango harvesting robot[J]. Journal of Field Robotics, 2025, 42(7): 3705-3725.
[19]	谯懿宸, 冯青春, 吴建伟, 等. 苹果采摘机械臂模块化构型优化设计[J]. 农业工程学报, 2026, 42(2): 40-51.
	QIAO Y C, FENG Q C, WU J W, et al. Optimization design of the modular configuration for apple picking manipulator[J]. Transactions of the Chinese Society of Agricultural Engineering, 2026, 42(2): 40-51.
[20]	李涛, 邱权, 赵春江, 等. 矮化密植果园多臂采摘机器人任务规划[J]. 农业工程学报, 2021, 37(2): 1-10.
	LI T, QIU Q, ZHAO C J, et al. Task planning of multi-arm harvesting robots for high-density dwarf orchards[J]. Transactions of the Chinese Society of Agricultural Engineering, 2021, 37(2): 1-10.
[21]	王宁, 韩雨晓, 王雅萱, 等. 农业机器人全覆盖作业规划研究进展[J]. 农业机械学报, 2022, 53(S1): 1-19.
	WANG N, HAN Y X, WANG Y X, et al. Research progress of agricultural robot full coverage operation planning[J]. Transactions of the Chinese Society for Agricultural Machinery, 2022, 53(S1): 1-19.
[22]	LI T, XIE F, QIU Q, et al. Multi-arm robot task planning for fruit harvesting using multi-agent reinforcement learning[C]// 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Piscataway, New Jersey, USA: IEEE, 2023: 4176-4183.
[23]	贾江鸣, 王翔, 周宇杰, 等. 双臂采茶机器人的协同采摘规划[J]. 茶叶科学, 2025, 45(4): 671-686.
	JIA J M, WANG X, ZHOU Y J, et al. Collaborative picking planning for a dual-arm tea-picking robot[J]. Journal of Tea Science, 2025, 45(4): 671-686.
[24]	YE Y, BAYAR G, VIDAL R, et al. Bin-Dog: A robotic platform for bin management in orchards[J]. Journal of Field Robotics, 2017, 34(6): 1230-1249.
[25]	HAYASHI S, YAMAMOTO S, TSUBOTA S, et al. Automation technologies for strawberry harvesting and packing operations in Japan[J]. Journal of Berry Research, 2014, 4(1): 19-27.
[26]	丁述勇, 张征, 丁文洁, 等. 多巷道式立体车库优化设计与车辆存取策略研究[J]. 工程设计学报, 2021, 28(4): 443-449.
	DING S Y, ZHANG Z, DING W J, et al. Optimization design of multi-lane stereo garage and research on vehicle access strategy[J]. Chinese Journal of Engineering Design, 2021, 28(4): 443-449.
[27]	LI Z K, MIAO L. Automated stereo-garage with multiple cache parking spaces—Structure, system and scheduling performance[J]. Automation in Construction, 2020, 119: 103377.
[28]	ZHU M T, LIU Z F, ZENG Y X, et al. Nordihydroguaiaretic acid reduces postharvest berry abscission in grapes[J]. Postharvest Biology and Technology, 2022, 183: 111748.
[29]	LI Z B, CHEN S Q, QI M, et al. Inhibition of postharvest Rachis browning of table grapes by sulfur dioxide: evidence from phenolic metabolism and sulfur assimilation[J]. Postharvest Biology and Technology, 2023, 204: 112413.
[30]	PARR B, LEGG M, ALAM F. Analysis of depth cameras for proximal sensing of grapes[J]. Sensors, 2022, 22(11): 4179.
[31]	CHEN L Y, NACERI A, SWIKIR A, et al. Autonomous and teleoperation control of a drawing robot avatar[C]// 2024 IEEE International Conference on Robotics and Automation (ICRA). Piscataway, New Jersey, USA: IEEE, 2024: 15714-15720.
[32]	LOU K R, WANG Z B, ZHANG B, et al. Analysis and experimentation on the motion characteristics of a dragon fruit picking robot manipulator[J]. Agriculture, 2024, 14(11): 2095.
[33]	LI Y L, FENG Q C, JI C, et al. GNSS and LiDAR integrated navigation method in orchards with intermittent GNSS dropout[J]. Applied Sciences, 2024, 14(8): 3231.

指标	5组采摘实验
指标	E1	E2	E3	F1	F2
采摘数量	50	50	50	50	50
采摘成功数量	48	46	47	41	42
采摘失败数量	2	4	3	9	8
准确识别率/%	100	100	100	100	100
精准定位率/%	98	94	96	94	98
采摘成功率/%	96	92	94	82	84
果簇漏采率/%	0	2	0	14	12
总体耗时/min	7.25	7.08	8.17	5.75	6.3
单簇平均耗时/s	8.7	8.5	9.8	6.9	7.6