Novel method for the visual navigation path detection of jujube harvester autopilot based on image processing

Xiongchu Zhang, Bingqi Chen, Jingbin Li, Xin Fang, Congli Zhang, Shubo Peng, Yongzheng Li

Abstract


To realize automatic harvesting of the jujube, the 4ZZ-4A2 full-hydraulic self-propelled jujube harvester was designed and manufactured. For achieving the jujube harvester autopilot, a novel algorithm for visual navigation path detection was proposed. The centerline of tree row lines was taken as the navigation path. The method included four main parts: image preprocessing, image segmentation, tree row lines access, and navigation path access. The methods of threshold segmentation, noise removal, and border smoothing were utilized on the image in Lab colors pace for the image segmentation. The least square method was employed to fit the tree row lines, and the centerline was obtained as the navigation path. Experimental results indicated that the average false detection rate was 3.98%, and the average detection speed was 41 fps. The algorithm meets the requirements of the jujube harvester autopilot in terms of accuracy and speed. It also can lay the foundation for accomplishing the jujube harvester vision-based autopilot.
Keywords: visual navigation path, jujube orchards, image processing, Lab color space, seeded region growing
DOI: 10.25165/j.ijabe.20231605.7638

Citation: Zhang X C, Chen B Q, Li J B, Fang X, Zhang C L, Peng S B, et al. Novel method for the visual navigation path detection of the jujube harvester autopilot based on image processing. Int J Agric & Biol Eng, 2023; 16(5): 189-197.

Keywords


visual navigation path, jujube orchards, image processing, Lab color space, seeded region growing

Full Text:

PDF

References


Sharifi M, Chen X. A novel vision based row guidance approach for navigation of agricultural mobile robots in orchards. In: 2015 6th International Conference on Automation, Robotics and Applications (ICARA), 2015; pp.251-255.

Raikwar S, Fehrmann J, Herlitzius T. Navigation and control development for a four-wheel-steered mobile orchard robot using model-based design. Computers and Electronics in Agriculture, 2022; 202: 107410.

Zhou L, Li J B, Ding L P, Ding H Z, Shi G K, Li D L. Design and evaluation of a mechanical floor-standing jujube picker. Agriculture, 2022; 12(8): 1-20.

Jiang H B, Xiao Y L, Zhang Y W, Wang X J, Tai H J. Curve path detection of unstructured roads for the outdoor robot navigation. Mathematical and computer modelling, 2013; 58(3-4): 536-544.

Torres-Sospedra J, Nebot P. A new approach to visual-based sensory system for navigation into orange groves, Sensors. 2011; 11(4): 4086-4103.

English A, Ross P, Ball D, Corke P. Vision based guidance for robot navigation in agriculture. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, 2014; pp.1693-1698.

Shalal N, Low T, McCarthy C, Hancock N. A review of autonomous navigation systems in agricultural environments. SEAg 2013: Innovative Agricultural Technologies for a Sustainable Future, Engineers Australia, 2013.

Kim K, Deb A, Cappelleri D J. P-AgBot: In-row and under-canopy agricultural robot for monitoring and physical sampling. IEEE Robotics and Automation Letters, 2022; 7(3): 7942-7949.

Bakker T, van Asselt K, Bontsema J, Müller J, van Straten G. Autonomous navigation using a robot platform in a sugar beet field. Biosystems Engineering, 2011; 109(4): 357-368.

Pini M, Marucco G, Falco G, Nicola M, De Wilde W. Experimental testbed and methodology for the assessment of RTK GNSS receivers used in precision agriculture. IEEE Access, 2020; 8: 14690-14703.

Rovira-Más F, Chatterjee I, Sáiz-Rubio V. The role of GNSS in the navigation strategies of cost-effective agricultural robots. Computers and electronics in Agriculture, 2015; 112: 172-183.

Nehme H, Aubry C, Solatges T, Savatier X, Boutteau R, Rossi R. Lidar-based structure tracking for agricultural robots: Application to autonomous navigation in vineyards. Journal of Intelligent & Robotic Systems, 2021; 103: 1-16.

Higuti V A H, Velasquez A E B, Magalhaes D V, Becker M, Chowdhary G. Under canopy light detection and ranging‐based autonomous navigation. Journal of Field Robotics, 2019; 36(3): 547-567.

Raj T, Hanim H F, Huddin A B, Ibrahim M F, Hussain A. A survey on LiDAR scanning mechanisms. Electronics, 2020; 9(5): 741.

Wang T, Chen B, Zhang Z, Li H, Zhang M. Applications of machine vision in agricultural robot navigation: A review. Computers and Electronics in Agriculture, 2022; 198: 107085.

Bonin-Font F, Ortiz A, Oliver G. Visual navigation for mobile robots: A survey. Journal of Intelligent and Robotic Systems, 2008; 53: 263-296.

Zhou M K, Xia J F, Yang F, Zheng K, Hu M J, Li D, et al. Design and experiment of visual navigated UGV for orchard based on Hough matrix and RANSAC. Int J Agric & Biol Eng, 2021; 14(6): 176–184. doi: 10.25165/j.ijabe.20211406.5953

Lyu H K, Park C-H, Han D H, Kwak S, Choi B. Orchard free space and center line estimation using Naive Bayesian classifier for unmanned ground self-driving vehicle. Symmetry, 2018; 10(9): 355.

Choi K H, Han S K, Han S H, Park K-H, Kim K-S, Kim S. Morphology-based guidance line extraction for an autonomous weeding robot in paddy fields. Computers and Electronics in Agriculture, 2015; 113: 266-274.

Jiang G, Wang Z, Liu H. Automatic detection of crop rows based on multi-ROIs. Expert systems with applications, 2015; 42(5): 2429-2441.

Chen J, Qiang H, Wu J, Xu G, Liu X. Extracting the navigation path of a tomato-cucumber greenhouse robot based on a median point Hough transform. Computers and Electronics in Agriculture, 2020; 174: 105472.

Gao G, Li M. Navigating path recognition for greenhouse mobile robot based on K-means algorithm. Transactions of the CSAE, 2014; 30(7): 25-33. (in Chinese)

Peng S, Kan Z, Li J. Extraction of visual navigation directrix for harvesting operation in short-stalked and close-planting jujube orchard. Transactions of the CSAE, 2017; 33(9): 45-52. (in Chinese)

Adams R, Bischof L. Seeded region growing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1994; 16(6): 641-647.

Zhou J, Geng S, Qiu Q, et al. A deep-learning extraction method for orchard visual navigation lines. Agriculture, 2022; 12(10): 1650.

Chen T. The past, present, and future of image and multidimensional signal processing. IEEE Signal Processing Magazine, 1998; 15(2): 21-58.

Bhattacharyya S. A brief survey of color image preprocessing and segmentation techniques. Journal of Pattern Recognition Research, 2011; 1(1): 120-129.

Vasuki P, Kanimozhi J, Devi M B. A survey on image preprocessing techniques for diverse fields of medical imagery. In: 2017 IEEE International Conference on Electrical, Instrumentation and Communication Engineering (ICEICE), IEEE, 2017; doi:10.1109/ICEICE.2017.8192443.

Wang D C C, Vagnucci A H, Li C C. Digital image enhancement: A survey. Computer Vision, Graphics, and Image Processing, 1983; 24(3): 363-381.

Ahirwal B, Khadtare M, Mehta R. FPGA based system for color space transformation RGB to YIQ and YCbCr. In: 2007 International Conference on Intelligent and Advanced Systems, IEEE, 2007; pp.1345-1349.

Liu Z, Chen W, Zou Y, Hu C. Regions of interest extraction based on HSV color space. In: IEEE 10th International Conference on Industrial Informatics, IEEE, 2012; pp.481-485. .doi:10.1109/indin.2012.6301214.

Asmare M H, Asirvadam V S, Iznita L. Color space selection for color image enhancement applications. In: 2009 International Conference on Signal Acquisition and Processing, IEEE, 2009; pp.208-212.

Jiang W J, Yang Z T, Wang P F, Cao Q X. Navigation path points extraction method based on color space and depth information for combine harvester. In: 2020 5th International Conference on Advanced Robotics and Mechatronics (ICARM), IEEE, 2020; pp.622-627. doi:10.1109/ICARM49381.2020.9195358.

Du Q, Wang D, Sha L. Recognition of mobile robot navigation path based on K-means algorithm. International Journal of Pattern Recognition and Artificial Intelligence, 2020; 34(8): 2059028.

Mao J, Cao Z, Wang H, Zhang B, Niu W. Agricultural robot navigation path recognition based on k-means algorithm for large-scale image segmentation. In: 2019 14th IEEE Conference on Industrial Electronics and Applications (ICIEA), IEEE, 2019; pp.1233-1237. doi:10.1109/ICIEA.2019.8834296.

Jin Y, Wen S, Shi Z, et al. Target recognition and navigation path optimization based on NAO robot. Applied Sciences, 2022; 12(17): 8466.

Loresco P J M, Dadios E P, Valenzuela I C. Color space analysis using KNN for lettuce crop stages identification in smart farm setup. In: TENCON 2018: IEEE Region 10 Conference, IEEE, 2018; pp.2040-2044. doi:10.1109/TENCON.2018.8650209.

Larijani M R, Asli‐Ardeh E A, Kozegar E, Loni R. Evaluation of image processing technique in identifying rice blast disease in field conditions based on KNN algorithm improvement by K‐means. Food Science & Nutrition, 2019; 7(12): 3922-3930.

Hossain E, Hossain M F, Rahaman M A. A color and texture based approach for the detection and classification of plant leaf disease using KNN classifier. In: 2019 International Conference on Electrical, Computer and Communication Engineering (ECCE), IEEE, 2019; doi:10.1109/ECACE.2019.8679247.

Weeks A R, Hague G E. Color segmentation in the HSI color space using the k-means algorithm. In: Nonlinear image processing VIII. SPIE, 1997; 3026: 143-154.

Senthilkumaran N, Vaithegi S. Image segmentation by using thresholding techniques for medical images. Computer Science & Engineering: An International Journal, 2016; 6(1): 1-13.

Woebbecke D M, Meyer G E, Von Bargen K, et al. Shape features for identifying young weeds using image analysis. Transactions of the ASAE, 1995; 38(1): 271-281.

Ribeiro A, Fernández-Quintanilla C, Barroso J, Garcíaalegre M C. Development of an image analysis system for estimation of weed pressure. Precision Agriculture, 2005; 5: 169-174.

Kataoka T, Kaneko T, Okamoto H, Hata S. Crop growth estimation system using machine vision. In: Proceedings 2003 IEEE/ASME international conference on advanced intelligent mechatronics (AIM 2003), IEEE, 2003; 2: b1079-b1083. doi:10.1109/AIM.2003.1225492.

Hague T, Tillett N D, Wheeler H. Automated crop and weed monitoring in widely spaced cereals. Precision Agriculture, 2006; 7: 21-32.

Meyer G E, Neto J C. Verification of color vegetation indices for automated crop imaging applications. Computers and electronics in agriculture, 2008; 63(2): 282-293.

Pratt W K. Digital image processing. Wiley-Interscience. 2007.

Jack K. Video demystified: a handbook for the digital engineer. Elsevier, 2011.

Otsu N. A threshold selection method from gray-level histograms. IEEE transactions on systems, man, and cybernetics, 1979; 9(1): 62-66.

Ramos A R, Gaspar P D. Algorithm for path recognition in-between tree rows for agricultural wheeled-mobile robots. International Journal of Mechanical and Mechatronics Engineering, 2019; 13(1): 34-37.

García-Santillán I, Guerrero J M, Montalvo M, Pajares G. Curved and straight crop row detection by accumulation of green pixels from images in maize fields. Precision Agriculture, 2018; 19: 18-41.

Zhang X Y, Li X N, Zhang B H, Zhou J, Tian G Z, Xiong Y J, et al. Automated robust crop-row detection in maize fields based on position clustering algorithm and shortest path method. Computers and Electronics in Agriculture, 2018; 154: 165-175.

Chen J Q, Qiang H, Wu J H, Xu G W. Navigation path extraction for greenhouse cucumber-picking robots using the prediction-point Hough transform. Computers and Electronics in Agriculture, 2021; 180: 105911.

Hough P V C. Method and means for recognizing complex patterns: U.S. Patent 3,069,654. 1962-12-18.

Winterhalter W, Fleckenstein F V, Dornhege C, Burgard W. Crop row detection on tiny plants with the pattern hough transform. IEEE Robotics and Automation Letters, 2018; 3(4): 3394-3401.

He B, Liu G, Ji Y, Si Y S, Gao R. Auto recognition of navigation path for harvest robot based on machine vision. In: Computer and Computing Technologies in Agriculture IV: 4th IFIP TC 12 Conference, CCTA 2010, Nanchang, China, October 22-25, 2010, Springer Berlin Heidelberg, 2011; 344: 138-148. doi:10.1007/978-3-642-18333-1_19.

Amin A, Fischer S. A document skew detection method using the Hough transform. Pattern Analysis & Applications, 2000; 3: 243-253.




Copyright (c) 2023 International Journal of Agricultural and Biological Engineering

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

2023-2026 Copyright IJABE Editing and Publishing Office