Localization Method for Agricultural Robots Based on Fusion of LiDAR and IMU

doi:10.12133/j.smartag.SA202401009

Abstract

Abstract:

[Objective] High-precision localization technology serves as the crucial foundation in enabling the autonomous navigation operations of intelligent agricultural robots. However, the traditional global navigation satellite system (GNSS) localization method faces numerous limitations, such as tree shadow, electromagnetic interference, and other factors in the agricultural environment brings challenges to the accuracy and reliability of localization technology. To address the deficiencies and achieve precise localization of agricultural robots independent of GNSS, a localization method was proposed based on the fusion of three-dimensional light detection and ranging (LiDAR) data and inertial measurement unit (IMU) information to enhance localization accuracy and reliability. [Methods] LiDAR was used to obtain point cloud data in the agricultural environment and realize self-localization via point cloud matching. By integrating real-time motion parameter measurements from the IMU with LiDAR data, a high-precision localization solution for agricultural robots was achieved through a specific fusion algorithm. Firstly, the LiDAR-obtained point cloud data was preprocessed and the depth map was used to save the data. This approach could reduce the dimensionality of the original LiDAR point cloud, and eliminate the disorder of the original LiDAR point cloud arrangement, facilitating traversal and clustering through graph search. Given the presence of numerous distinct crops like trees in the agricultural environment, an angle-based clustering method was adopted. Specific angle-based clustering criteria were set to group the point cloud data, leading to the segmentation of different clusters of points, and obvious crops in the agricultural environment was effectively perceived. Furthermore, to improve the accuracy and stability of positioning, an improved three-dimensional normal distribution transform (3D-NDT) localization algorithm was proposed. This algorithm operated by matching the LiDAR-scanned point cloud data in real time with the pre-existing down sampled point cloud map to achieve real-time localization. Considering that direct down sampling of LiDAR point clouds in the agricultural environment could result in the loss of crucial environmental data, a point cloud clustering operation was used in place of down sampling operation, thereby improving matching accuracy and positioning precision. Secondly, to address potential constraints and shortcomings of using a single sensor for robot localization, a multi-sensor information fusion strategy was deployed to improve the localization accuracy. Specifically, the extended Kalman filter algorithm (EKF) was chosen to fuse the localization data from LiDAR point cloud and the IMU odometer information. The IMU provided essential motion parameters such as acceleration and angular velocity of the agricultural robot, and by combining with the LiDAR-derived localization information, the localization of the agricultural robot could be more accurately estimated. This fusion approach maximized the advantages of different sensors, compensated for their individual limitations, and improved the overall localization accuracy of the agricultural robot. [Results and Discussions] A series of experimental results in the Gazebo simulation environment of the robot operating system (ROS) and real operation scenarios showed that the fusion localization method proposed had significant advantages. In the simulation environment, the average localization errors of the proposed multi-sensor data fusion localization method were 1.7 and 1.8 cm, respectively, while in the experimental scenario, these errors were 3.3 and 3.3 cm, respectively, which were significantly better than the traditional 3D-NDT localization algorithm. These findings showed that the localization method proposed in this study could achieve high-precision localization in the complex agricultural environment, and provide reliable localization assistance for the autonomous functioning of agricultural robots. [Conclusions] The proposed localization method based on the fusion of LiDAR data and IMU information provided a novel localization solution for the autonomous operation of agricultural robots in areas with limited GNSS reception. Through the comprehensive utilization of multi-sensor information and adopting advanced data processing and fusion algorithms, the localization accuracy of agricultural robots could be significantly improved, which could provide a new reference for the intelligence and automation of agricultural production.

Key words: agricultural robots, LiDAR localization, point cloud matching, extended Kalman filter, sensors fusion

LIU Yang, JI Jie, PAN Deng, ZHAO Lijun, LI Mingsheng. Localization Method for Agricultural Robots Based on Fusion of LiDAR and IMU[J]. Smart Agriculture, 2024, 6(3): 94-106.

Figures/Tables 19

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Fig. 9

Fig. 10

Fig. 11

Fig. 12

Table 1

Fig. 13

Fig. 14

Fig. 15

Fig. 16

Fig. 17

Table 2

References 27

1	YIN X, WANG Y X, CHEN Y L, et al. Development of autonomous navigation controller for agricultural vehicles[J]. International journal of agricultural and biological engineering, 2020, 13(4): 70-76.
2	柳景斌, 毛井锋, 吕海霞, 等. BDS/GPS组合定位可靠性分析与粗差探测研究[J]. 武汉大学学报(信息科学版), 2023, 48(2): 214-223.
	LIU J B, MAO J F, LYU H X, et al. Reliability analysis and gross error detection of BDS/GPS combined positioning[J]. Geomatics and information science of Wuhan university, 2023, 48(2): 214-223.
3	YOUSUF S, KADRI M B. Information fusion of GPS, INS and odometer sensors for improving localization accuracy of mobile robots in indoor and outdoor applications[J]. Robotica, 2021, 39(2): 250-276.
4	THANPATTRANON P, AHAMED T, TAKIGAWA T. Navigation of autonomous tractor for orchards and plantations using a laser range finder: Automatic control of trailer position with tractor[J]. Biosystems engineering, 2016, 147: 90-103.
5	BLOK P M, VAN BOHEEMEN K, VAN EVERT F K, et al. Robot navigation in orchards with localization based on Particle filter and Kalman filter[J]. Computers and electronics in agriculture, 2019, 157(C): 261-269.
6	UNDERWOOD J P, JAGBRANT G, NIETO J I, et al. LiDAR-based tree recognition and platform localization in orchards[J]. Journal of field robotics, 2015, 32(8): 1056-1074.
7	BESL P J, MCKAY N D. Method for registration of 3-D shapes[C]// Proc SPIE 1611, sensor fusion IV: Control paradigms and data structures. Burlingame, California, USA: SPIE, 1992, 1611: 586-606.
8	BIBER P, STRASSER W. The normal distributions transform: a new approach to laser scan matching[C]// Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453. Piscataway, JerseyNew, USA: IEEE, 2003, 3: 2743-2748.
9	SAARINEN J, ANDREASSON H, STOYANOV T, et al. Normal Distributions Transform Occupancy Maps: Application to large-scale online 3D mapping[C]// 2013 IEEE International Conference on Robotics and Automation. Piscataway, New Jersey, USA: IEEE, 2013: 2233-2238.
10	HU F J, REN T J, SHI S B. Discrete point cloud registration using the 3D normal distribution transformation based newton iteration[J]. Journal of multimedia, 2014, 9(7): ID 934.
11	WANG Q S, ZHANG J. Point cloud registration algorithm based on combination of NDT and PLICP[C]// 2019 15th International Conference on Computational Intelligence and Security (CIS). Piscataway, New Jersey, USA: IEEE, 2019
12	AKAI N, MORALES L Y, TAKEUCHI E, et al. Robust localization using 3D NDT scan matching with experimentally determined uncertainty and road marker matching[C]// 2017 IEEE Intelligent Vehicles Symposium (IV). Piscataway, New Jersey, USA: IEEE, 2017: 1356-1363.
13	KALLASI F, RIZZINI D L, CASELLI S. Fast keypoint features from laser scanner for robot localization and mapping[J]. IEEE robotics and automation letters, 2016, 1(1): 176-183.
14	金泰宇, 黄劲松. 一种基于语义地图的激光雷达定位方法[J]. 测绘地理信息, 2022, 47(4): 28-32.
	JIN T Y, HUANG J S. A LiDAR positioning methodology based on semantic map[J]. Journal of geomatics, 2022, 47(4): 28-32.
15	LEVINSON J, THRUN S. Robust vehicle localization in urban environments using probabilistic maps[C]// 2010 IEEE International Conference on Robotics and Automation. Piscataway, New Jersey, USA: IEEE, 2010: 4372-4378.
16	HATA A, WOLF D. Road marking detection using LiDAR reflective intensity data and its application to vehicle localization[C]// 17th International IEEE Conference on Intelligent Transportation Systems (ITSC). Piscataway, New Jersey, USA: IEEE, 2014: 584-589.
17	WAN G W, YANG X L, CAI R L, et al. Robust and precise vehicle localization based on multi-sensor fusion in diverse city scenes[C]// 2018 IEEE International Conference on Robotics and Automation (ICRA). Piscataway, New Jersey, USA: IEEE, 2018: 4670-4677.
18	邓泽武, 胡钊政, 周哲, 等. 融合激光雷达与双层地图模型的智能车定位[J]. 汽车工程, 2022, 44(7): 1018-1026.
	DENG Z W, HU Z Z, ZHOU Z, et al. Intelligent vehicle positioning by fusing LiDAR and double-layer map model[J]. Automotive engineering, 2022, 44(7): 1018-1026.
19	崔文, 薛棋文, 李庆玲, 等. 基于三维点云地图和ESKF的无人车融合定位方法[J]. 工矿自动化, 2022, 48(9): 116-122.
	CUI W, XUE Q W, LI Q L, et al. Unmanned vehicle fusion positioning method based on 3D point cloud map and ESKF[J]. Journal of mine automation, 2022, 48(9): 116-122.
20	张文玥, 娄小平, 陈福笛. 移动机器人多传感器融合定位仿真研究[J]. 计算机仿真, 2023, 40(3): 436-441.
	ZHANG W Y, LOU X P, CHEN F D. Research on multi-sensor fusion localization simulation of mobile robot[J]. Computer simulation, 2023, 40(3): 436-441.
21	涂远泯, 刘飞飞, 曾波华, 等. 基于多传感器融合技术的移动机器人位姿估计方法研究[J]. 制造业自动化, 2023, 45(11): 137-141.
	TU Y M, LIU F F, ZENG B H, et al. Research on pose estimation method of mobile robot based on multi-sensor fusion technology[J]. Manufacturing automation, 2023, 45(11): 137-141.
22	BOGOSLAVSKYI I, STACHNISS C. Fast range image-based segmentation of sparse 3D laser scans for online operation[C]// 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). New York, USA: ACM, 2016.
23	杨奇峰, 曲道奎, 徐方. 基于3D-NDT的移动机器人定位算法研究[J]. 控制工程, 2020, 27(4): 613-619.
	YANG Q F, QU D K, XU F. Localization algorithm of outdoor mobile robot based on 3D-NDT[J]. Control engineering of China, 2020, 27(4): 613-619.
24	MILIOTO A, VIZZO I, BEHLEY J, et al. RangeNet ++: Fast and accurate LiDAR semantic segmentation[C]// 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). New York, USA: ACM, 2019: 4213-4220.
25	KALMAN R E. A new approach to linear filtering and prediction problems[J]. Journal of basic engineering, 1960, 82(1): 35-45.
26	GALATI R, REINA G, MESSINA A, et al. Survey and navigation in agricultural environments using robotic technologies[C]// 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS). Piscataway, New Jersey, USA: IEEE, 2017: 1-6.
27	YAN Y X, ZHANG B H, ZHOU J, et al. Real-time localization and mapping utilizing multi-sensor fusion and visual-IMU-wheel odometry for agricultural robots in unstructured, dynamic and GPS-denied greenhouse environments[J]. Agronomy, 2022, 12(8): ID 1740.

路径	算法	X方向上的平均误差绝对值/cm	Y方向上的平均误差绝对值/cm	X方向上的最大误差绝对值/cm	Y方向上的最大误差绝对值/cm
转向路径	3D-NDT定位算法	5.8	3.7	11.7	9.9
转向路径	EKF融合定位算法	1.7	2.4	4.9	5.6
矩形路径	3D-NDT定位算法	6.2	3.2	15.3	10.0
矩形路径	EKF融合定位算法	1.7	1.1	4.6	4.7
综合两种工况	3D-NDT定位算法	6.0	3.5	15.3	10.0
综合两种工况	EKF融合定位算法	1.7	1.8	4.9	5.6
算法性能提升/%		71.7	48.6

路径	算法	X方向上的平均误差绝对值/cm	Y方向上的平均误差绝对值/cm	X方向上的最大误差绝对值/cm	Y方向上的最大误差绝对值/cm
转向路径	3D-NDT定位算法	6.7	6.0	11.7	11.1
转向路径	EKF融合定位算法	3.5	3.9	7.5	10.5
矩形路径	3D-NDT定位算法	7.3	5.0	16.8	13.2
矩形路径	EKF融合定位算法	3.0	2.6	8.4	8.7
综合两种工况	3D-NDT定位算法	7.0	5.5	16.8	13.2
综合两种工况	EKF融合定位算法	3.3	3.3	8.4	10.5
算法性能提升/%		52.9	40.0

[1]	ZHOU Longgang, LIU Ting, LU Jinzhu. Traversal Path Planning for Farmland in Hilly Areas Based on Floyd and Improved Genetic Algorithm [J]. Smart Agriculture, 2023, 5(4): 45-57.
[2]	GUO Wei, WU Huarui, ZHU Huaji. Design and Application of Facility Greenhouse Image Collecting and Environmental Data Monitoring Robot System [J]. Smart Agriculture, 2020, 2(3): 48-60.