Detection of multi-class coconut clusters for robotic picking under occlusion conditions

Authors

  • Yuxing Fu 1. School of Information and Communication Engineering, Hainan University, Haikou 570228, China 2. School of Mechanical and Electrical Engineering, Hainan University, Haikou 570228, China
  • Hongcheng Zheng School of Mechanical and Electrical Engineering, Hainan University
  • Zongbin Wang School of Mechanical and Electrical Engineering, Hainan University
  • Jinyang Huang School of Mechanical and Electrical Engineering, Hainan University
  • Wei Fu 1. School of Information and Communication Engineering, Hainan University, Haikou 570228, China 2. School of Mechanical and Electrical Engineering, Hainan University, Haikou 570228, China

DOI:

https://doi.org/10.25165/ijabe.v18i1.9031

Keywords:

coconut clusters, picking robot, leaves-occluded, multi-class detection, YOLOv7-tiny

Abstract

With the development of tree-climbing robots and robotic end-effectors, it is possible to develop automated coconutpicking robots with the help of machine vision technology. Coconuts grow in clusters in the canopy and are easily occluded by leaves. Therefore, the detection of multi-class coconut clusters according to the occlusion condition is necessary for robots to develop picking strategies. The coconut detection model, named YOLO-Coco, was developed based on the YOLOv7-tiny network. It detected coconuts in different conditions such as not-occluded, leaves-occluded, and trunk-occluded fruit. The developed model used Efficient Channel Attention (ECA) to enhance the feature weights extracted by the backbone network. Re-parameterization Convolution (RepConv) made the model convolution layers deeper and provided more semantic information for the detection head. Finally, the Bi-directional Feature Pyramid Network (BiFPN) was used to optimize the head network structure of YOLO-Coco to achieve the balanced fusion of multi-scale features. The results showed that the mean average precision (mAP) of YOLO-Coco for detecting multi-class coconut clusters was 93.6%, and the average precision (AP) of not-occluded, leaves-occluded, and trunk-occluded fruit were 90.5%, 93.8%, and 96.4%, respectively. The detection accuracy of YOLO-Coco for yellow coconuts was 5.1% higher than that for green coconuts. Compared with seven mainstream deep learning networks, YOLO-Coco achieved the highest detection accuracy in detecting multi-class coconut clusters, while maintaining advantages in detection speed and model size. The developed model can accurately detect coconuts in complex canopy environments, providing technical support for the visual system of coconut-picking robots. Keywords: coconut clusters, picking robot, leaves-occluded, multi-class detection, YOLOv7-tiny DOI: 10.25165/j.ijabe.20251801.9031 Citation: Fu Y X, Zheng H C, Wang Z B, Huang J Y, Fu W. Detection of multi-class coconut clusters for robotic picking under occlusion conditions. Int J Agric & Biol Eng, 2025; 18(1): 267–278.

References

Mat K, Abdul Kari Z, Rusli N D, Che Harun H, Wei L S, Rahman M M, et al. Coconut palm: Food, feed, and nutraceutical properties. Animals, 2022; 12(16): 2107.

Ru S F, Wang J F, Fan J Q. Design and parameter optimization of removing coconut fiber device by bionic steel wire roller brush based on characteristics of claw-toe. Transactions of the CSAE, 2018; 22(34): 27–35. (in Chinese)

Hainan Provincial Bureau of Statistics. Hainan Statistical Yearbook, 2023. Available: https://stats.hainan.gov.cn/tjj/tjsu/ndsj/2023/202311/P020231129587527396831.pdf. Accessed on [2023-12-02].

Caladcad J A, Cabahug S, Catamco M R, Villaceran P E, Cosgafa L, Cabizares K N, et al. Determining Philippine coconut maturity level using machine learning algorithms based on acoustic signal. Computers and Electronics in Agriculture, 2020; 172: 105327.

Ignacio I F, Miguel T S. Research opportunities on the coconut (Cocos nucifera L.) using new technologies. South African Journal of Botany, 2021; 141: 414–420.

Zhou H Y, Wang X, Au W, Kang H W, Chen C. Intelligent robots for fruit harvesting: Recent developments and future challenges. Precision Agriculture, 2022; 23(5): 1856–1907.

Gené-Mola J, Vilaplana V, Rosell-Polo J R, Morros J R, Ruiz-Hidalgo J, Gregorio E. Multi-modal deep learning for Fuji apple detection using RGB-D cameras and their radiometric capabilities. Computers and Electronics in Agriculture, 2019; 162: 689–698.

Megalingam R K, Manoharan S K, Mohandas S M, Vadivel S R R, Gangireddy R, Ghanta S, et al. Amaran: An unmanned robotic coconut tree climber and harvester. IEEE-ASME Transactions on Mechatronics, 2021; 26(1): 288–299.

Wibowo T S, Sulistijono I A, Risnumawan A. End-to-end coconut harvesting robot. In: 2016 International Electronics Symposium (IES), 2016; pp.444–449. doi: 10.1109/ELECSYM.2016.7861047.

Parvathi S, Selvi S T. Detection of maturity stages of coconuts in complex background using Faster R-CNN model. Biosystems Engineering, 2021; 202: 119–132.

Fu L S, Majeed Y, Zhang X, Karkee M, Zhang Q. Faster R-CNN-based apple detection in dense-foliage fruiting-wall trees using RGB and depth features for robotic harvesting. Biosystems Engineering, 2020; 197: 245–256.

Tang Y C, Zhou H, Wang H J, Zhang Y Q. Fruit detection and positioning technology for a Camellia oleifera C. Abel orchard based on improved YOLOv4-tiny model and binocular stereo vision. Expert Systems with Applications, 2023; 211: 118573.

Zhu X Y, Chen F J, Zhang X W, Zheng Y L, Peng X D, Chen C. Detection the maturity of multi-cultivar olive fruit in orchard environments based on Olive-EfficientDet. Scientia Horticulturae, 2024; 324: 112607.

Patrício D I, Rieder R. Computer vision and artificial intelligence in precision agriculture for grain crops: A systematic review. Computers and Electronics in Agriculture, 2018; 153: 69–81.

Shaikh T A, Rasool T, Lone F R. Towards leveraging the role of machine learning and artificial intelligence in precision agriculture and smart farming. Computers and Electronics in Agriculture, 2022; 198: 107119.

Zhu X Y, Li J J, Jia R C, Liu B, Yao Z H, Yuan A H, et al. LAD-Net: A novel light weight model for early apple leaf pests and diseases classification. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2023; 20(2): 1156–1169.

Wang Z, Hua Z X, Wen Y C, Zhang S J, Xu X S, Song H B. E-YOLO: Recognition of estrus cow based on improved YOLOv8n model. Expert Systems with Applications, 2024; 238: 122212.

Montoya-Cavero L E, Torres R D D, Gómez-Espinosa A, Cabello J A E. Vision systems for harvesting robots: Produce detection and localization. Computers and Electronics in Agriculture, 2022; 192: 106562.

Yang Y Y, Han Y X, Li S, Yang Y D, Zhang M, Li H. Vision based fruit recognition and positioning technology for harvesting robots. Computers and Electronics in Agriculture, 2023; 213: 108258.

Zhang B, Wang R R, Zhang H M, Yin C H, Xia Y Y, Fu M, et al. Dragon fruit detection in natural orchard environment by integrating lightweight network and attention mechanism. Frontiers in Plant Science, 2022; 13: 1040923.

Zhang J, Karkee M, Zhang Q, Zhang X, Yaqoob M, Fu L S, et al. Multi-class object detection using faster R-CNN and estimation of shaking locations for automated shake-and-catch apple harvesting. Computers and Electronics in Agriculture, 2020; 173: 105384.

Gao F, Fu L S, Zhang X, Majeed Y, Li R, Karkee M, Zhang Q. Multi-class fruit-on-plant detection for apple in SNAP system using Faster R-CNN. Computers and Electronics in Agriculture, 2020; 176: 105634.

Song H B, Shang Y Y, He D J. Review on deep learning technology for fruit target recognition. Transactions of the CSAM, 2023; 54(1): 1–19. (in Chinese)

Peng H X, Huang B, Shao Y Y, Li Z S, Zhang C W, Chen Y, et al. General improved SSD model for picking object recognition of multiple fruits in natural environment. Transactions of the CSAE, 2018; 16(34): 155–162. (in Chinese)

Redmon J, Divvala S, Girshick R, Farhadi A. You Only Look Once: Unified, real-time object detection. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas: IEEE, 2016; pp.779–788. doi: 10.1109/CVPR.2016.91.

Li T H, Sun M, He Q H, Zhang G S, Shi G, Ding X, et al. Tomato recognition and location algorithm based on improved YOLOv5. Computers and Electronics in Agriculture, 2023; 208: 107759.

Xu L J, Wang Y H, Shi X S, Tang Z L, Chen X Y, Wang Y C, et al. Real-time and accurate detection of citrus in complex scenes based on HPL-YOLOv4. Computers and Electronics in Agriculture, 2023; 205: 107590.

Suo R, Gao F F, Zhou Z X, Fu L S, Song Z Z, Dhupia J, et al. Improved multi-classes kiwifruit detection in orchard to avoid collisions during robotic picking. Computers and Electronics in Agriculture, 2021; 182: 106052.

Zhang F, Chen Z J, Ali S, Yang N, Fu S L, Zhang Y K. Multi-class detection of cherry tomatoes using improved YOLOv4-Tiny. Int J Agric & Biol Eng, 2023; 16(2): 225–231.

Wang C Y, Bochkovskiy A, Liao H Y M. YOLOv7: Trainable Bag-of-Freebies Sets New State-of-the-Art for Real-Time Object Detectors. In: 2023 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023; pp.7464–7475. doi: 10.1109/CVPR52729.2023.00721.

Wang Y, Yao X Z, Li B, Xu S, Yi Z F, Zhao J H. Malformed sweet pepper fruit identification algorithm based on improved YOLOv7-tiny. Transactions of the CSAM, 2023; 11(54): 236–246. (in Chinese)

Wang Q L, Wu B G, Zhu P F, Li P H, Zuo W M, Hu Q H. ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks. In: 2020 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020; pp.11531–11539. doi: 10.1109/CVPR42600.2020.01155.

Li Q M, Han Z C, Wu X M. Deeper insights into graph convolutional networks for semi-supervised learning. In: 32nd AAAI Conference on Artificial Intelligence, 2018; pp.3538–3545. doi: 10.48550/arXiv.1801.07606.

Ding X H, Zhang X Y, Ma N N, Han J G, Ding G G, Sun J. RepVGG: Making VGG-style ConvNets Great Again. In: 2021 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021; pp.13728–13737. doi: 10.1109/CVPR46437.2021.01352.

Tan M X, Pang R Y, Le Q V. EfficientDet: Scalable and Efficient Object Detection. In: 2020 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020; pp.10778–10787. doi: 10.1109/CVPR42600.2020.01079.

Yao T, Tan Z P, Cheng E, Wu L G. Method for daylily detection and segmentation based on improved YOLOv7-seg. Transactions of the CSAE, 2024; 40(9): 146–153. (in Chinese)

Divyanth L G, Soni P, Pareek C M, Machavaram R, Nadimi M, Paliwal J, et al. Detection of coconut clusters based on occlusion condition using attention-guided Faster R-CNN for robotic harvesting. Foods, 2022; 11(23): 3903.

Downloads

Published

2025-03-07

How to Cite

Fu, Y., Zheng, H., Wang, Z., Huang, J., & Fu, W. (2025). Detection of multi-class coconut clusters for robotic picking under occlusion conditions. International Journal of Agricultural and Biological Engineering, 18(1), 267–278. https://doi.org/10.25165/ijabe.v18i1.9031

Issue

Vol. 18 No. 1 (2025): IJABE

Section

Information Technology, Sensors and Control Systems

License

IJABE is an international peer reviewed open access journal, adopting Creative Commons Copyright Notices as follows.


Authors who publish with this journal agree to the following terms:


  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.

  2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.

  3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).