Synchronous detection method for litchi fruits and picking points of a litchi-picking robot based on improved YOLOv8-pose

Authors

  • Hongxing Peng 1. College of Mathematics and Informatics, South China Agricultural University, Guangzhou 510642, China; 2. Foshan-Zhongke Innovation Research Institute of Intelligent Agriculture and Robotics, Foshan528251, Guangdong, China; 3. Key Laboratory of Smart Agricultural Technology in Tropical South China, Ministry of Agriculture and Rural Affairs, Guangzhou 510642, China;
  • Qijun Liang 1. College of Mathematics and Informatics, South China Agricultural University, Guangzhou 510642, China;
  • Xiangjun Zou 2. Foshan-Zhongke Innovation Research Institute of Intelligent Agriculture and Robotics, Foshan528251, Guangdong, China; 4. College of Intelligent Manufacturing and Modern Industry, Xinjiang University, Urumqi 830046, Xinjiang, China;
  • Hongjun Wang 2. Foshan-Zhongke Innovation Research Institute of Intelligent Agriculture and Robotics, Foshan528251, Guangdong, China; 5. College of Engineering, South China Agricultural University, Guangzhou 510642, China
  • Juntao Xiong 1. College of Mathematics and Informatics, South China Agricultural University, Guangzhou 510642, China; 2. Foshan-Zhongke Innovation Research Institute of Intelligent Agriculture and Robotics, Foshan528251, Guangdong, China;
  • Yanlin Luo 1. College of Mathematics and Informatics, South China Agricultural University, Guangzhou 510642, China;
  • Shangkun Guo 1. College of Mathematics and Informatics, South China Agricultural University, Guangzhou 510642, China;
  • Guanjia Shen 1. College of Mathematics and Informatics, South China Agricultural University, Guangzhou 510642, China;

DOI:

https://doi.org/10.25165/ijabe.v18i4.9303

Keywords:

litchi, object detection, picking point detection, YOLOv8-pose, picking robot

Abstract

In the unstructured litchi orchard, precise identification and localization of litchi fruits and picking points are crucial for litchi-picking robots. Most studies adopt multi-step methods to detect fruit and locate picking points, which are slow and struggle to cope with complex environments. This study proposes a YOLOv8-iGR model based on YOLOv8n-pose improvement, integrating end-to-end network for both object detection and key point detection. Specifically, this study considers the influence of auxiliary points on picking point and designs four litchi key point strategies. Secondly, the architecture named iSaE is proposed, which combines the capabilities of CNN and attention mechanism. Subsequently, C2f is replaced by Generalized Efficient Layer Aggregation Network (GELAN) to reduce model redundancy and improve detection accuracy. Finally, based on RFAConv, RFAPoseHead is designed to address the issue of parameter sharing in large convolutional kernels, thereby more effectively extracting feature information. Experimental results demonstrate that YOLOv8iGR achieves AP of 95.7% in litchi fruit detection, and the Euclidean distance error of picking points is less than 8 pixels across different scenes, meeting the requirements of litchi picking. Additionally, the GFLOPs of the model is reduced by 10.71%. The accuracy of the model’s localization for picking points was tested through field picking experiments. In conclusion, YOLOv8iGR exhibits outstanding detection performance along with lower model complexity, making it more feasible for implementation on robots. This will provide technical support for the vision system of the litchi-picking robot. Keywords: litchi, object detection, picking point detection, YOLOv8-pose, picking robot DOI: 10.25165/j.ijabe.20251804.9303 Citation: Peng H X, Liang Q J, Zou X J, Wang H J, Xiong J T, Luo Y L, et al. Synchronous detection method for litchi fruits and picking points of a litchi-picking robot based on improved YOLOv8-pose. Int J Agric & Biol Eng, 2025; 18(4): 266–274.

References

Xie J X, Peng J J, Wang J X, Chen B H, Jing T W, Sun D Z, et al. Litchi detection in a complex natural environment using the YOLOv5-litchi model. Agronomy, 2022; 12(12): 3054.

Qi X K, Dong J S, Lan Y B, Zhu H. Method for identifying litchi picking position based on YOLOv5 and PSPNet. Remote Sensing, 2022; 14(9): 2004.

Zhang G M, Cao H, Hu K W, Pan Y Q, Deng Y Q, Wang H J, et al. Accurate cutting-point estimation for robotic lychee harvesting through geometry-aware learning. arXiv: 2404.00364, 2024; In press. doi: 10.48550/arXiv.2404.00364.

Tang Y C, Qiu J J, Zhang Y Q, Wu D X, Cao Y H, Zhao K X, et al. Optimization strategies of fruit detection to overcome the challenge of unstructured background in field orchard environment: A review. Precision Agriculture, 2023; 24(4): 1183–1219.

Peng H X, Zhong J R, Liu H, Li J, Yao M W, Zhang X. ResDense-focalDeepLabV3+ enabled litchi branch semantic segmentation for robotic harvesting. Computers and Electronics in Agriculture, 2023; 206: 107691.

Zheng C, Chen P F, Pang J, Yang X F, Chen C X, Tu S Q, et al. A mango picking vision algorithm on instance segmentation and key point detection from RGB images in an open orchard. Biosystems Engineering, 2021; 206: 32–54.

Du X Q, Meng Z C, Ma Z H, Lu W W, Cheng H C. Tomato 3D pose detection algorithm based on keypoint detection and point cloud processing. Computers and Electronics in Agriculture, 2023; 212: 108056.

Narayanan M. SENetV2: Aggregated dense layer for channelwise and global representations. arXiv: 2311.10807. 2023; In Press. doi: 10.48550/ arXiv.2311.10807.

Hu J, Shen L, Albanie S, Sun G, Wu E H. Squeeze-and-excitation networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020; 42(8): 2021–2023.

Zhang J N, Li X T, Li J, Liu L, Xue Z C, Zhang B S. Rethinking mobile block for efficient attention-based models. In: 2023 IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France: IEEE, 2023; pp.1389–1400.

Sandler M, Howard A, Zhu M, Zhmoginov A, Chen L-C. Mobilenetv2: Inverted residuals and linear bottlenecks. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA: IEEE, 2018; pp.4510–4520.

Wang C-Y, Yeh I-H, Liao H-Y M. YOLOv9: Learning what you want to learn using programmable gradient information. arXiv: 2402.13616, 2024; In press. doi: 10.48550/arXiv.2402.13616.

Wang C-Y, Liao H-Y M, Wu Y-H, Chen P-Y, Hsieh J-W, Yeh I-H. CSPNet: A new backbone that can enhance learning capability of CNN. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle, WA, USA: IEEE, 2020; 1571–1580.

Wang C-Y, Liao H-Y M, Yeh I-H. Designing network design strategies through gradient path analysis. arXiv: 2211.04800, 2022; In press. doi: 10.48550/arXiv.2211.04800.

Zhang X, Liu C, Yang D G, Song T T, Ye Y C, Li K, et al. Rfaconv: Innovating spatital attention and standard convolutional operation. arXiv: 2304.03198, 2023; In press. doi: 10.48550/arXiv.2304.03198.

Maji D, Nagori S, Mathew M, Poddar D. Yolo-pose: Enhancing yolo for multi person pose estimation using object keypoint similarity loss. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), New Orleans, LA, USA: IEEE, 2022; pp.2636– 2645. doi: 10.1109/CVPRW56347.2022.00297.

Xiong J T, He Z L, Lin R, Liu Z, Bu R B, Yang Z G, et al. Visual positioning technology of picking robots for dynamic litchi clusters with disturbance. Computers and Electronics in Agriculture, 2018; 151: 226– 237.

Zhuang J W, Hou C J, Tang Y, He Y, Guo Q W, Zhong Z Y, et al. Computer vision-based localisation of picking points for automatic litchi harvesting applications towards natural scenarios. Biosystems Engineering, 2019; 187: 1–20.

Downloads

Published

2025-08-21

How to Cite

Peng, H., Liang, Q., Zou, X., Wang, H., Xiong, J., Luo, Y., … Shen, G. (2025). Synchronous detection method for litchi fruits and picking points of a litchi-picking robot based on improved YOLOv8-pose. International Journal of Agricultural and Biological Engineering, 18(4), 266–274. https://doi.org/10.25165/ijabe.v18i4.9303

Issue

Vol. 18 No. 4 (2025): IJABE

Section

Information Technology, Sensors and Control Systems

License

IJABE is an international peer reviewed open access journal, adopting Creative Commons Copyright Notices as follows.


Authors who publish with this journal agree to the following terms:


  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.

  2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.

  3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).