Superpixel-based segmentation algorithm for mature citrus

Qinghua Yang, Yiqin Chen, Yi Xun, Guanjun Bao

Abstract


With the decrease of agricultural labors and the increase in production costs, harvesting robots have become a research hotspot in recent years. To guide harvesting robots to pick mature citrus more precisely under variable illumination conditions, an image segmentation algorithm based on superpixel was proposed. Efficient simple linear iterative clustering (SLIC) algorithm which takes similarity of adjacent pixels into account was adopted to segment the images captured under variable illumination conditions into superpixels. The color and texture features of these superpixels were extracted and fused into feature vectors as descriptors to train backpropagation neural networks (BPNN) classifier in the next step. The adjacency information of superpixels was considered by calculating the global-local binary pattern (LBP) in R component images when extracting texture features. To accelerate the classification process, the mean of Cr-Cb image was utilized to find superpixels of interest which were regarded as candidates of citrus superpixels. These candidates were then classified by a pre-trained BPNN model with superpixel-level accuracy of 98.77% and pixel-level accuracy of 94.96%, while the average time to segment one image was 0.4778 s. Therefore, the results indicated that a superpixel-based segmentation algorithm toward citrus images had decent light robustness as well as high accuracy that could guide harvesting robot to pick mature citrus efficiently.
Keywords: superpixel, image segmentation, BPNN, variable illumination, mature citrus
DOI: 10.25165/j.ijabe.20201304.5607

Citation: Yang Q H, Chen Y Q, Xun Y, Bao G J. Superpixel-based segmentation algorithm for mature citrus. Int J Agric & Biol Eng, 2020; 13(4): 166–171.

Keywords


superpixel, image segmentation, BPNN, variable illumination, mature citrus

Full Text:

PDF

References


Zhang S B, Bao G J, Yang Q H, Chen L. Study on mechanical properties of citrus bases on robotic harvesting. Journal of Zhejiang University of Technology, 2012; 40(3): 340–344.

Sanders K F. Orange harvesting systems review. Biosystems Engineering, 2005; 90(2): 115–125.

Gongal A, Amatya S, Karkee M, Zhang Q, Lewis K. Sensors and systems for fruit detection and localization: A review. Computers and Electronics in Agriculture, 2015; 116: 8–19.

Fathallah, Fadi A. Musculoskeletal disorders in labor-intensive agriculture. Applied Ergonomics, 2010; 41(6): 738–743.

Li P L, Lee S H, Hsu H Y. Review on fruit harvesting method for potential use of automatic fruit harvesting systems. International Conference on Power Electronics and Engineering Application, 2011; 23: 351–366.

Feng Q C, Cheng W, Zhou J J, Wang X. Design of structured-light vision system for tomato harvesting robot. Int J Agric & Biol Eng, 2014; 7(2): 19–26.

Sun G X, Li Y B, Wang X C, Hu G Y, Wang X, Zhang Y. Image segmentation algorithm for greenhouse cucumber canopy under various natural lighting conditions. Int J Agric & Biol Eng, 2016; 9(3): 130–138.

Zhang Y J, Deng L, Li M Z. Estimation of citrus yield based on imageprocessing. Transactions of the CSAM, 2009; 40(9): 97–99. (in Chinese)

Cai J R, Zhou X J, Li Y L, F J. Recognition of mature oranges in natural scene based on machine vision. Transactions of the CSAE, 2008; 24(1): 175–178. (in Chinese)

Li Y, Yang C H, Hu Y C, Zhang H, Yang Y. Overlapping citrus target recognition and localization method based on convex shell and distance transformation. Modern Manufacturing Engineering, 2018; (9): 82–87. (in Chinese)

Zhao D A, Liu X Y, Chen Y, Ji W, Jia W K, Hu C L. Image recognition at night for apple picking robot. Transactions of the CSAM, 2015; 46(3): 15–22. (in Chinese).

Lu J, Sang N. Detecting citrus fruits and occlusion recovery under natural illumination conditions. Computers and Electronics in Agriculture, 2005; 110:121–130.

Lü Q, Cai J R, Liu B, Deng L, Zhang Y J. Identification of fruit and branch in natural scenes for citrus harvesting robot using machine vision and support vector machine. Int J Agric & Biol Eng, 2014; 7(2): 115–121.

Lin G C, Tang Y C, Zou X J, Li J H, Xiong J T. In-field citrus detection and localization based on RGB-D image analysis. Biosystems Engineering, 2019; 186: 34–44.

Liu T H, Ehsani R, Toudeshki A, Zou X J, Wang H J. Detection of citrus fruit and tree trunks in natural environments using a multi-elliptical boundary model. Computers in Industry, 2018; 99: 9–16.

Zhuang J J, Luo S M, Hou C J, Tang Y, He Y, Xue X Y. Detection of orchard citrus fruits using a monocular machine vision-based method for automatic fruit picking applications. Computers and Electronics in Agricultures, 2018; 152: 64–73.

Bansal R, Lee W S, Satish S. Green citrus detection using fast Fourier transform (FFT) leakage. Precision Agriculture, 2013; 14(1): 59–70.

Sengupta S, Lee W S. Identification and determination of the number of immature green citrus fruit in a canopy under different ambient light conditions. Biosystems Engineering, 2014; 117: 51–61.

Wang C L, Lee W S, Zou X J, Choi D, Gan H, Diamond J. Detection and counting of immature green citrus fruit based on the local binary patterns (LBP) feature using illumination-normalized images. Precision Agriculture, 2018; 19(6): 1062–1083.

Lu J, Lee W S, Gan H, Hu X W. Immature citrus fruit detection based on local binary pattern feature and hierarchical contour analysis. Biosystems Engineering, 2018; 171: 78–90.

Song X Y, Zhou L L, Li Z G, Chen J, Zeng L, Yan B. Review on superpixel methods in image segmentation. Journal of Image and Graphics, 2015; 20(5): 599–608.

Yang F, Lu H C, Yang M H. Robust superpixel tracking. IEEE Transaction on Image Process, 2014; 23(4): 1639–1651.

Jiang J J, Ma J Y, Chen C, Wang Z Y, Cai Z H, Wang L Z. SuperPCA: A superpixelwise PCA approach for unsupervised feature extraction of hyperspectral imagery. IEEE Transactions on Geoscience and Remote Sensing, 2018; 56(8): 4581–4593.

Zhou J, Zhu J R, Wang M J. Pear orchard scene segmentation based on conditional random fields. Transactions of the CSAM, 2015; 46(2): 8–13. (in Chinese)

Xu W Y, Tian G Z, Ji C Y, Zhang B, Jiang S J, Zhang C. FSLIC superpixel segmentation algorithm for apple image in natural scene. Transactions of the CSAM, 2016; 47(9): 1–10, 28. (in Chinese)

Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S. SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012; 34(11): 2274–2282.

Liu M Y, Tuzel O, Ramalingam S, Chellappa R. Entropy rate superpixel segmentation. IEEE International Conference on Computer Vision and Pattern Recognition, 2011; pp.2097–2104.

Yang Q H, Luo S L, Chang C, Xun Y, Bao G J. Segmentation algorithm for Hangzhou white chrysanthemums based on least squares support vector machine. Int J Agric & Biol Eng, 2019; 12(4): 127–134.

Gao D Q. On structures of supervised linear basis function feedforward three-layered neural networks. Chinese J. Computers, 1998; 21(1): 80–86. (in Chinese)

Sun Y P, Liu M, Wu C. Intelligent prediction algorithm of trim beam number in colored textile manufacturing process. Control Engineering of China, 2005; 12(6): 523–526. (in Chinese)

Shen X L, Zhang J S. Research of intrusion detection based on the BP networks and the improved PSO algorithm. Computer Engineering & Science, 2010; 32(6): 34–36, 73. (in Chinese)




Copyright (c) 2020 International Journal of Agricultural and Biological Engineering

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

2023-2026 Copyright IJABE Editing and Publishing Office