Accurate and rapid image segmentation method for bayberry automatic picking via machine learning

Huan Lei, Chentong Li, Yu Tang, Zhengyu Zhong, Zeyu Jiao

Abstract


Due to the short ripening period and complex picking environment, bayberry generally relies on mechanical equipment for picking, especially the automatic picking system guided by vision. Thus, it is crucial to locate the bayberry in the view accurately and rapidly. Although efforts have been made, the existing methods are difficult to implement due to the limited amount of data and the processing speed. In this study, an accurate and rapid segmentation method based on machine learning was proposed to address this problem. First, the images collected by the visual guidance system were pre-processed by contrast-limited adaptive histogram equalization (CLAHE) based on the Y component of the YUV color space. Taking advantage of the color difference map of RB and RG for the segmentation of different colors, an adaptive color difference map foreground segmentation method was then adopted for bayberry region foreground segmentation. Finally, distance transforms and marking control watershed methods were exploited to achieve single bayberry fruit segmentation. Furthermore, with the help of the convex hull theory and fruit shape characteristics, the irregular background interference areas were filtered out, which improved the accuracy of bayberry segmentation performance. The experimental results show that this method can achieve better segmentation of bayberry in complex orchard environment with an accuracy of 97.4% and only takes 0.136 s to calculate once.
Key words: bayberry, image segmentation, machine learning, automatic picking, computer vision
DOI: 10.25165/j.ijabe.20231606.6834

Citation: Lei H, Li C T, Tang Y, Zhong Z Y, Jiao Z Y. Accurate and rapid image segmentation method for bayberry automatic picking via machine learning. Int J Agric & Biol Eng, 2023; 16(6): 246–254.

Keywords


bayberry, image segmentation, machine learning, automatic picking, computer vision

Full Text:

PDF

References


Qiu Y Y, Shen B, Guo X Z, Xie X B, Zheng X L, Qi X J, et al. Effects of foliar nutrition on leaf photosynthesis and fruit quality of Chinese bayberry. Journal of Fruit Science, 2009; 26（6）: 902–906. （in Chinese）

Fang Z X, Bhandari, B. Comparing the efficiency of protein and maltodextrin on spray drying of bayberry juice. Food Research International, 2012; 48（2）: 478–483.

Linker R, Cohen O, Naor A. Determination of the number of green apples in RGB images recorded in orchards. Computers and Electronics in Agriculture, 2012; 81: 45–57.

Wei X Q, Jia K, Lan J H, Li Y W, Zeng Y L, Wang C M. Automatic method of fruit object extraction under complex agricultural background for vision system of fruit picking robot. Optik, 2014; 125（19）: 5684–5689.

Wang C L, Tang Y C, Zou X J, SiTu W M, Feng W X. A robust fruit image segmentation algorithm against varying illumination for vision system of fruit harvesting robot. Optik, 2017; 131: 626–631.

Xu L M, Lyu J D. Recognition method for apple fruit based on SUSAN and PCNN. Multimedia Tools and Applications, 2018; 77: 7205–7219.

Septiarini A, Hamdani H, Hatta H R, Anwar K. Automatic image segmentation of oil palm fruits by applying the contour-based approach. Scientia Horticulturae, 2020; 261: 108939.

Lu J, Lee W S, Gan H, Hu X W. Immature citrus fruit detection based on local binary pattern feature and hierarchical contour analysis. Biosystems Engineering, 2018; 171: 78–90.

Si Y S, Liu G, Feng J. Location of apples in trees using stereoscopic vision. Computers and Electronics in Agriculture, 2015; 112: 68–74.

Xu L M. , He K R, Lyu J D. Bayberry image segmentation based on manifold ranking salient object detection method. Biosystems Engineering, 2019; 178: 264–274.

He Z L, Xiong J T, Chen S M, Li Z X, Chen S F, Zhong Z, et al. A method of green citrus detection based on a deep bounding box regression forest. Biosystems Engineering, 2020; 193: 206–215.

Bargoti S, Underwood J P. Image segmentation for fruit detection and yield estimation in apple orchards. Journal of Field Robotics, 2017; 34（6）: 1039–1060.

Li Q W, Jia W K, Sun M L, Hou S J, Zheng Y J. A novel green apple segmentation algorithm based on ensemble U-Net under complex orchard environment. Computers and Electronics in Agriculture, 2021; 180: 105900.

Yu Y, Zhang K L, Yang L, Zhang D X. Fruit detection for strawberry harvesting robot in non-structural environment based on Mask-RCNN. Computers and Electronics in Agriculture, 2019; 163: 104846.

Flores P, Zhang Z, Igathinathane C, Jithin M, Naik D, Stenger J, et al. Distinguishing seedling volunteer corn from soybean through greenhouse color, color-infrared, and fused images using machine and deep learning. Industrial Crops and Products, 2021; 161: 113223.

Reza A M. Realization of the contrast limited adaptive histogram equalization （CLAHE） for real-time image enhancement. Journal of VLSI Signal Processing Systems for Signal, Image and Video Technology, 2004; 38（1）: 35–44.

Otsu N. A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man, and Cybernetics, 1979; 9（1）: 62–66.

Burtsev S V, Kuzmin Y P. An efficient flood-filling algorithm. Computers & Graphics, 1993; 17（5）: 549–561.

Dorj U O, Lee M, Yun S S. An yield estimation in citrus orchards via fruit detection and counting using image processing. Computers and Electronics in Agriculture, 2017; 140: 103–112.

Jung C, Kim C. Segmenting clustered nuclei using H-minima transform-based marker extraction and contour parameterization. IEEE Transactions on Biomedical Engineering, 2010; 57（10）: 2600–2604.

Wang D D, Song H B, Tie Z H, Zhang W Y, He D J. Recognition and localization of occluded apples using K-means clustering algorithm and convex hull theory: A comparison. Multimedia Tools and Applications, 2016; 75（6）: 3177–3198.

Lyu J D, Wang F, Xu L M, Ma Z H, Yang B. A segmentation method of bagged green apple image. Scientia Horticulturae, 2019; 246: 411–417.

Guo Q W, Chen Y Y, Tang Y, Zhuang J J, He Y, Hou C J, et al. Lychee fruit detection based on monocular machine vision in orchard environment. Sensors, 2019; 19（19）: 4091.

Zhao D A, Liu X Y, Chen Y, Ji W, Jia W K, Hu C L. Image recognition at night for apple picking robot. Transactions of the CSAM, 2015; 46（3）: 15–22.

Fu L S, Wang B, Cui Y J, Su S, Yoshinori G, Kobayashi T. Kiwifruit recognition at nighttime using artificial lighting based on machine vision. In J Agric & Biol Eng, 2015; 8（4）: 52–59.

Zhuang Z M, Li N, Joseph Raj A. N, Mahesh V G V, Qiu S M. An RDAU-NET model for lesion segmentation in breast ultrasound images. PloS One, 2019; 14（8）: e0221535.

Zhuang J J. Luo S M, Hou C J, Tang Y, He Y, Xue X Y. Detection of orchard citrus fruits using a monocular machine vision-based method for automatic fruit picking applications. Computers and Electronics in Agriculture, 2018; 152: 64–73.




Copyright (c) 2023 International Journal of Agricultural and Biological Engineering

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

2023-2026 Copyright IJABE Editing and Publishing Office