Key technologies of tomato-picking robots based on machine vision
DOI:
https://doi.org/10.25165/ijabe.v%25vi%25i.8954Keywords:
tomato-picking robot, end effector, machine vision, deep learning, YOLO v8nAbstract
To address the challenges of harsh harvesting environments, high labor intensity, and low picking efficiency in tomato harvesting, this study investigates the key technologies related to the end-effector design, detection and recognition, and spatial localization of tomato-picking robots. A non-contact cavity-type end-effector is designed, which effectively prevents tomato damage caused by compression during picking while preserving the peduncle. Additionally, the motion of the robotic arm is simulated for performance analysis. Subsequently, tomato images are captured and annotated for training deep neural network models. Both the original YOLO v8n and the improved YOLO v8n models are used for tomato image detection, with a focus on the impact of varying light intensities and different tomato maturities on recognition and localization accuracy. Experimental results demonstrate that the robot’s vision system achieves optimal recognition and localization performance under light intensities ranging from 20 000 to 30 000 lx, with an accuracy of 91.5%, an average image detection speed of 15.1 ms per image, and an absolute localization error of 1.55 cm. Furthermore, the prototype tomato-picking robot’s end-effector successfully performed stable grasping of individual tomatoes without damaging the skin, achieving a picking success rate of 83.3%, with an average picking time of approximately 9.5 s per fruit. This study provides a technical support for the automated harvesting of tomato-picking robots. Keywords: tomato-picking robot, end effector, machine vision, deep learning, YOLO v8n DOI: 10.25165/j.ijabe.20251803.8954 Citation: Yin Z R, Li H, Zuo Z J, Guan Z X. Key technologies of tomato-picking robots based on machine vision. Int J Agric & Biol Eng, 2025; 18(3): 247–256.References
Li J M, Xiang C Y, Wang X X, Guo Y M, Huang Z J. Current status and prospects of China’s tomato industry during the 13th Five-Year Plan. China Vegetables, 2021;（2）: 13-20. （in Chinese）
Zhou M, Li C B. The current status and prospects of tomato seed industry in China. Vegetables, 2022;（5）: 6-10. （in Chinese）
Hu H R, Zhang Y Y, Zhou J L, Chen Q, Wang J P. Research status of end effectors for fruit and vegetable harvesting robots. Chinese Journal of Agricultural Machinery, 2024; 45（4）: 231-236. （in Chinese）
Zhang P, Tang Q Y, Chen L F, Qin C L, Wang L Y. Research status and progress analysis of fruit and vegetable picking robots. Southern Agricultural Machinery, 2023; 54（7）: 9-12.
Molaei F, Ghatrehsamani S. Kinematic-based multi-objective design optimization of a grapevine pruning robotic manipulator. AgriEngineering, 2022; 4（3）: 606-625.
Moreira G, Magalhães S A, Pinho T, dos Santos F S, Cunha M. Benchmark of deep learning and a proposed HSV colour space models for the detection and classification of greenhouse tomato. Agronomy, 2022; 12（2）: 356.
Huang C W, Cai D X, Wang W Z, Li J, Duan J L, Yang Z. Development of an automatic control system for a hydraulic pruning robot. Computers and Electronics in Agriculture, 2023; 214: 108329.
Zhao Q, Li L J, Wu Z C, Guo X, Li J. Optimal design and experiment of manipulator for camellia pollen picking. Applied Sciences, 2022; 12（16）: 8011.
Chen R. Application and development trends of mechanical automation technology in modern agriculture. Agricultural Engineering Technology, 2024; 44（2）: 64-65. （in Chinese）
Li T H, Sun M, He Q H, Zhang G S, Shi G Y, Ding X M, et al. Tomato recognition and location algorithm based on improved YOLOv5. Computers and Electronics in Agriculture, 2023; 208: 107759.
Kondo N, Yata K, Iida M, Shiigi T, Monta M, Kurita M, et al. Development of an end-effector for a tomato cluster harvesting robot. Engineering in Agriculture, Environment and Food, 2010; 3（1）: 20-24.
Liu J Z, Peng Y, Faheem M. Experimental and theoretical analysis of fruit plucking patterns for robotic tomato harvesting. Computers and Electronics in Agriculture, 2020; 173: 105330.
Liu J Z, Yuan Y, Gao Y, Tang S Q, Li Z G. Virtual model of grip-and-cut picking for simulation of vibration and falling of grape clusters. Transactions of the ASABE, 2019; 62（3）: 603-614.
Yaguchi H, Nagahama K, Hasegawa T, Inaba M. Development of an autonomous tomato harvesting robot with rotational plucking gripper. In: 2016 IEEE/RSJ international conference on intelligent robots and systems （IROS）, Daejeon, Korea （South）: IEEE, 2016; pp.652-657.
Wang X N, Wu P H, Feng Q C, Wang G H. Design and experiments of tomato picking robot system. Agricultural Mechanization Research, 2016; 38（4）: 94-98. （in Chinese）
Zheng X J, Rong J C, Zhang Z Q, Yang Y, Li W, Yuan T. Fruit growing direction recognition and nesting grasping strategies for tomato harvesting robots. Journal of Field Robotics, 2023; 41（2）: 300-313.
Rong J C, Wang P B, Wang T J, Hu L, Yuan T. Fruit pose recognition and directional orderly grasping strategies for tomato harvesting robots. Computers and Electronics in Agriculture, 2022; 202: 107430.
Fujinaga T, Yasukawa S, Ishii K. Development and evaluation of a tomato fruit suction cutting device. In: 2021 IEEE/SICE International Symposium on System Integration （SII）, Iwaki, Fukushima, Japan: IEEE, 2021; pp.628-633. doi: 10.1109/IEEECONF49454.2021.9382670.
Liang X F, Jin C Q, Ni M D, Wang Y W. Acquisition and experiment on location information of picking point of tomato fruit clusters. Transactions of the Chinese Society of Agricultural Engineering, 2018; 34（16）: 163-169. （in Chinese）
Feng J H, Li Z W, Rong Y L, Sun Z L. Identification of mature tomatoes based on an algorithm of modified circular Hough transform. Chinese Journal of Agricultural Machinery and Chemistry, 2021; 42（4）: 190-196. （in Chinese）
Li H, Tao H X, Cui L H, Liu D W, Sun J T, Zhang M. Tomato fruit recognition and localization method based on SOM-K-means algorithm. Transactions of the Chinese Society for Agricultural Machinery, 2021; 52（1）: 23-29. （in Chinese）
Wang J P, Xu G. Research on tomato maturity detection method based on machine vision and electronic nose fusion. Food and Machinery, 2022; 38（02）: 148-152.
Benavides M, CantónGarbín M, SánchezMolina J A, Rodríguez F. Automatic tomato and peduncle location system based on computer vision for use in robotized harvesting. Applied Sciences, 2020; 10（17）: 5887.
Fass E, Shlomi E, Ziv C, Glikman O, Helman D. Machine learning models based on hyperspectral imaging for pre-harvest tomato fruit quality monitoring. Computers and Electronics in Agriculture, 2025; 229: 109788. doi: 10.1016/J.COMPAG.2024. 109788.
Bai Y H, Mao S H, Zhou J, Zhang B H. Clustered tomato detection and picking point location using machine learning-aided image analysis for automatic robotic harvesting. Precision Agriculture, 2023; 24（2）: 727-743.
Rong J C, Zhou H, Zhang F, Yuan T, Wang P B. Tomato cluster detection and counting using improved YOLOv5 based on RGB-D fusion. Computers and Electronics in Agriculture, 2023; 207: 107741.
Zhang R C, Zhou Y C, Hou Y H, Liu Z Y, Zhao H G, Zhao Y H. Counting tomatoes with different maturities using ultra-depth masking and improved YOLOv8. Transactions of the Chinese Society of Agricultural Engineering, 2024; 40（24）: 146-156. （in Chinese）
Wang X W, Liu J. Tomato anomalies detection in greenhouse scenarios based on YOLO-Dense. Frontiers in Plant Science, 2021; 12: 634103.
Li T H, Sun M, Ding X M, Li Y H, Zhang G S, Shi G Y, et al. T Tomato recognition method at the ripening stage based on YOLO v4 and HSV. Transactions of the Chinese Society of Agricultural Engineering, 2021; 37（21）: 183-190. （in Chinese）
Wang X F, Wu Z W, Jia M, Xu T, Pan C L, Qi X B, et al. Lightweight SM-YOLOv5 tomato fruit detection algorithm for plant factory. Sensors, 2023; 23（6）: 3336.
Cai Y Q, Cui B, Deng H, Zeng Z, Wang Q C, Lu D J, et al. Cherry tomato detection for harvesting using multimodal perception and an improved YOLOv7-tiny neural network. Agronomy, 2024; 14（10）: 2320.
Miao R H, Li Z W, Wu J L. Lightweight cherry tomato maturity detection method based on improved YOLO v7. Transactions of the Chinese Society of Agricultural Machinery, 2023; 54（10）: 225-233. （in Chinese）
Huang Y P, Liu Y, Yang Y T, Zhang Z W, Chen K J. Assessment of tomato color by spatially resolved and conventional vis NIR spectroscopies. Spectroscopy and Spectral Analysis, 2019; 39（11）: 3585-3591. （in Chinese）.
Zhang Z L L, He T T, Li Z W, Shi K L, Liu C B, Zheng W G. Quantitative grading method for tomato maturity using regional brightness correction. Transactions of the Chinese Society of Agricultural Engineering, 2023; 39（7）: 195-204. doi: 10.11975/j.issn. 1002-6819.202211192. （in Chinese）
Downloads
Published
How to Cite
Issue
Section
License
IJABE is an international peer reviewed open access journal, adopting Creative Commons Copyright Notices as follows.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
