Method for the fruit tree recognition and navigation in complex environment of an agricultural robot

Xiaolin Xie, Yuchao Li, Lijun Zhao, Xin Jin, Shengsheng Wang, Xiaobing Han

Abstract


To realize the visual navigation of agricultural robots in the complex environment of orchards, this study proposed a method for fruit tree recognition and navigation based on YOLOv5. The YOLOv5s model was selected and trained to identify the trunks of the left and right rows of fruit trees; the quadratic curve was fitted to the bottom center of the fruit tree recognition box, and the identified fruit trees were divided into left and right columns by using the extreme value point of the quadratic curve to obtain the left and right rows of fruit trees; the straight-line equation of the left and right fruit tree rows was further solved, the median line of the two straight lines was taken as the expected navigation path of the robot, and the path tracing navigation experiment was carried out by using the improved LQR control algorithm. The experimental results show that under the guidance of the machine vision system and guided by the improved LQR control algorithm, the lateral error and heading error can converge quickly to the desired navigation path in the four initial states of [0 m, −0.34 rad], [0.10 m, 0.34 rad], [0.15 m, 0 rad] and [0.20 m, −0.34 rad]. When the initial speed was 0.5 m/s, the average lateral error was 0.059 m and the average heading error was 0.2787 rad for the navigation trials in the four different initial states. Its average driving was 5.3 m into the steady state, the average value of steady state lateral error was 0.0102 m, the average value of steady state heading error was 0.0253 rad, and the average relative error of the robot driving along the desired navigation path was 4.6%. The results indicate that the navigation algorithm proposed in this study has good robustness, meets the operational requirements of robot autonomous navigation in orchard environment, and improves the reliability of robot driving in orchard.
Key words: fruit tree recognition, visual navigation, YOLOv5, complex environments, orchards
DOI: 10.25165/j.ijabe.20241702.8031

Citation: Xie X L, Li Y C, Zhao L J, Jin X, Wang S S, Han X B. Method for the fruit tree recognition and navigation in complex environment of an agricultural robot. Int J Agric & Biol Eng, 2024; 17(2): 221–229.

Keywords


fruit tree recognition, visual navigation, YOLOv5, complex environments, orchards

Full Text:

PDF

References


Wang S S, Geng L X. Development situation and countermeasures of agricultural mechanization in hilly and mountain areas. Agricultural Engineering, 2016; 6(5): 1–4. (in Chinese)

Zhang P, Zhang L N, Liu D, Wu H X, Jiao B. Research status of agricultural robot technology. Agricultural Engineering, 2019; 9(10): 1–12. (in Chinese)

Yan Q T, Li L X, Qiu Q, Zhang Z G, Meng Z J. Research status and development trend of small-mobile agricultural robots. Journal of Chinese Agricultural Mechanization, 2019; 40(5): 178–186. (in Chinese)

Nagle M, Intani K, Romano G, Mahayothee B, Sardsud V, Müller J. Determination of surface color of ‘all yellow’mango cultivars using computer vision. Int J Agric & Biol Eng, 2016; 9(1): 42–50.

Bi S, Wang Y H. Inter-line pose estimation and fruit tree location method for orchard robot. Transactions of the CSAM, 2021; 52(8): 16–26, 39. (in Chinese)

Pedersen S M, Fountas S, Sørensen C G, Van Evert F K, Blackmore B S. Chapter Robotic seeding: Economic perspectives. In: Precision Agriculture: Technology and Economic Perspectives, Springer, 2017; pp.167–179.

Zhang L, Zhang H D, Chen Y D, Dai S H, Li X M, Imou K J, Liu Z H, Li M. Real-time monitoring of optimum timing for harvesting fresh tea leaves based on machine vision. Int J Agric & Biol Eng, 2019; 12(1): 6–9.

Steward B, Gai J Y, Tang L. The use of agricultural robots in weed management and control. Burleigh Dodds Science Publishing, 2019; pp.161–186.

Zhang M J. Design of obstacle avoidance system of orchard lawn mower based on binocular vision technology. Master’s dissertation. Baoding: Hebei Agricultural University, 2019; 63p. (in Chinese)

Liu B, Yang C H, Xiong L Y, Wang K, Wang Y. Path recognition method of picking robot based on orchard natural environment. Jiangsu Journal of Agricultural Sciences, 2019; 35(5): 1222–1231. (in Chinese)

Mao F D. Strcture design and vision navigation research on mobile platform of picking robot in the hills and mountains. Master’s dissertation. Chongqing: Chongqing University of Technology, 2017; 103p. (in Chinese)

Liu H, Zhu S H, Shen Y, Tang J H. Trunk fast segmentation algorithm based on multi-feature fusion. Transactions of the CSAM, 2020; 51(1): 221–229. (im Chinese

Shalal N, Low T, McCarthy C, Hancock N. Orchard mapping and mobile robot localisation using on-board camera and laser scanner data fusion-Part A: Tree detection. Computers and Electronics in Agriculture, 2015; 119: 267–278.

Zeng F, Wang C, Ge S S. A survey on visual navigation for artificial agents with deep reinforcement learning. IEEE Access, 2020; 8: 135426–135442.

Han Z H, Li J, Yuan Y W, Fang X F, Zhao B, Zhu L C. Path recognition of orchard visual navigation based on U-Net. Transactions of the CSAM, 2021; 52(1): 30–39. (in Chinese)

Bonadies S, Gadsden S A. An overview of autonomous crop row navigation strategies for unmanned ground vehicles. Engineering in Agriculture, Environment and Food, 2019; 12(1): 24–31.

Radcliffe J, Cox J, Bulanon D M. Machine vision for orchard navigation. Computers in Industry, 2018; 98: 165–171.

Sharifi M, Chen X Q. A novel vision based row guidance approach for navigation of agricultural mobile robots in orchards. 2015 6th International Conference on Automation, Robotics and Applications (ICARA). IEEE, 2015; pp.251–255.

Yang S J, Mei S L, Zhang Y E. Detection of maize navigation centerline based on machine vision. IFAC-PapersOnLine, 2018; 51(17): 570–575.

Yi S, Li X R, Wu Z J, Zhu J M, Yuan X S. Night hare detection method based on infrared thermal imaging and improved YOLOv3. Transactions of the CSAE, 2019; 35(19): 223–229. (in Chinese)

Zhang W K, Sun H, Chen X K, Li X B, Yao L G, Dong H. Research on weed detection in vegetable seedling field based on improved YOLOv5. Journal of Graphics, 2023; 44(2): 346–356. (in Chinese)

Sivakumar A N, Modi S, Gasparino M V, Ellis C, Velasquez A E B, Chowdhary G, et al. Learned visual navigation for under-canopy agricultural robots. Agricultural Robots, 2021; arXiv: 2107. 02792, 2021.

Kurtulmuş F, Alibaş İ, Kavdır I. Classification of pepper seeds using machine vision based on neural network. Int J Agric & Biol Eng, 2016; 9(1): 51–62.




Copyright (c) 2024 International Journal of Agricultural and Biological Engineering

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

2023-2026 Copyright IJABE Editing and Publishing Office