Method for detecting 2D grapevine winter pruning location based on thinning algorithm and Lightweight Convolutional Neural Network

Qinghua Yang, Yuhao Yuan, Yiqin Chen, Yi Xun

Abstract


In viticulture, there is an increasing demand for automatic winter grapevine pruning devices, for which detection of pruning location in vineyard images is a necessary task, susceptible to being automated through the use of computer vision methods. In this study, a novel 2D grapevine winter pruning location detection method was proposed for automatic winter pruning with a Y-shaped cultivation system. The method can be divided into the following four steps. First, the vineyard image was segmented by the threshold two times Red minus Green minus Blue (2R−G−B) channel and S channel; Second, extract the grapevine skeleton by Improved Enhanced Parallel Thinning Algorithm (IEPTA); Third, find the structure of each grapevine by judging the angle and distance relationship between branches; Fourth, obtain the bounding boxes from these grapevines, then pre-trained MobileNetV3_small×0.75 was utilized to classify each bounding box and finally find the pruning location. According to the detection experiment result, the method of this study achieved a precision of 98.8% and a recall of 92.3% for bud detection, an accuracy of 83.4% for pruning location detection, and a total time of 0.423 s. Therefore, the results indicated that the proposed 2D pruning location detection method had decent robustness as well as high precision that could guide automatic devices to winter prune efficiently.
Keywords: grapevine winter pruning, Lightweight Convolutional Neural Network, thinning algorithm, detection method
DOI: 10.25165/j.ijabe.20221503.6750

Citation: Yang Q H, Yuan Y H, Chen Y Q, Xun Y. Method for detecting 2D grapevine winter pruning location based on thinning algorithm and Lightweight Convolutional Neural Network. Int J Agric & Biol Eng, 2022; 15(3): 177–183.

Keywords


grapevine winter pruning, Lightweight Convolutional Neural Network, thinning algorithm, detection method

Full Text:

PDF

References


Shamshiri R R, Weltzien C, Hameed I A, Yule I J, Grift T E, Balasundram S K, et al. Research and development in agricultural robotics: A perspective of digital farming. Int J Agric & Biol Eng, 2018; 11(4): 1–14.

Liu S, Whitty M. Automatic grape bunch detection in vineyards with an SVM classifier. Journal of Applied Logic, 2015; 13(4): 643–653.

Botterill T, Green R, Mills S. Finding a vine's structure by bottom-up parsing of cane edges. In: 2013 28th International Conference on Image and Vision Computing New Zealand (IVCNZ 2013), Wellington, New Zealand: IEEE, 2013; pp.112–117. doi: 10.1109/IVCNZ.2013.6727001.

Diago M-P, Correa C, Millán B, Barreiro P, Valero C, Tardaguila J. Grapevine yield and leaf area estimation using supervised classification methodology on RGB images taken under field conditions. Sensors, 2012; 12(12): 16988–17006.

Xu S, Xun Y, Jia T M, Yang Q H. Detection method for the buds on winter vines based on computer vision. In: 2014 Seventh International Symposium on Computational Intelligence and Design, Hangzhou: IEEE, 2014; pp.44–48. doi: 10.1109/ISCID.2014.26

Rosenfeld A. A characterization of parallel thinning algorithms. Information and Control, 1975; 29(3): 286–291.

Harris C G, Stephens M J. A combined corner and edge detector. Alvey Vision Conference, 1988; pp.147–152. doi: 10.5244/c.2.23.

Pérez D S, Bromberg F, Diaz C A. Image classification for detection of winter grapevine buds in natural conditions using scale-invariant features transform, bag of features and support vector machines. Computers and Electronics in Agriculture, 2017; 135: 81–95.

Lowe D G. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004; 60(2): 91–110.

Csurka G, Dance C R, Fan L, Willamowski J, Bray C. Visual categorization with bags of keypoints. In: Workshop on Statistical Learning in Computer Vision ECCV 2004, 2004; pp.1–22.

Vapnik V N, Mulier F M. Statistical learning theory. In: Learning from Data: Concepts, Theory, and Methods, IEEE, 2007; pp.99–150.

Díaz C A, Pérez D S, Miatello H, Bromberg F. Grapevine buds detection and localization in 3D space based on structure from motion and 2D image classification. Computers in Industry, 2018; 99: 303–312.

Zabawa L, Kicherer A, Klingbeil L, Toepfer R, Kuhlmann H, Roscher R. Counting of grapevine berries in images via semantic segmentation using convolutional neural networks. ISPRS Journal of Photogrammetry and Remote Sensing, 2020; 164: 73–83.

Palacios F, Bueno G, Salido J, Diago M P, Hernandez I, Tardaguila J. Automated grapevine flower detection and quantification method based on computer vision and deep learning from on-the-go imaging using a mobile sensing platform under field conditions. Computers and Electronics in Agriculture, 2020; 178: 105796. doi: 10.1016/j.compag.2020.105796.

Cruz A, Ampatzidis Y, Pierro R, Materazzi A, Panattoni A, Bellis L D, et al. Detection of grapevine yellows symptoms in Vitis vinifera L. with artificial intelligence. Computers and Electronics in Agriculture, 2019; 157: 63–76.

Marset W V, Perez D S, Diaz C A, Bromberg F. Towards practical 2D grapevine bud detection with fully convolutional networks. Computers and Electronics in Agriculture, 2021; 182: 105947. doi: 10.1016/j.compag.2020.105947.

Zhang T Y, Suen C Y. A fast parallel algorithm for thinning digital patterns. Communications of the ACM, 1984; 27(3): 236–239.

Naccache N J, Shinghal R. An investigation into the skeletonization approach of Hilditch. Pattern Recognition, 1984; 17(3): 279–284.

Lu H E, Wang P S P. A comment on “A fast parallel algorithm for thinning digital patterns”. Communications of the ACM, 1986; 29(3): 239–242.

Lu H E, Wang P S P. An improved fast parallel thinning algorithm for digital patterns. In: Proceedings CVPR '85: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1985; pp.364–367.

Zhao D D, Wang H B, Tao L, Zhou J. Improved EPTA parallel thinning algorithm. Computer Engineering and Applications, 2016; 52(9): 196–201. (in Chinese)

Sandler M, Howard A, Zhu M L, Zhmoginov A, Chen L-C. MobileNetV2: Inverted residuals and linear bottlenecks. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA: IEEE, 2018; pp.4510–4520. doi: 10.1109/ CVPR.2018.00474.

Howard A, Sandler M, Chu G, Chen L-C, Chen B, Tan M, et al. Searching for MobileNetV3. In: 2019 IEEE/CVF International Conference on Computer Vision, IEEE, 2019; pp.1314–1324.

Ma N, Zhang X, Zheng H-T, Sun J. ShuffleNet V2: Practical guidelines for efficient CNN architecture design. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y (Eds). Computer Vision - ECCV 2018, Springer, 2018; pp. 122–138. doi: 10.1007/978-3-030-01264-9_8.

Mingxing T, Le Q V. EfficientNet: Rethinking model scaling for Convolutional Neural Networks. arXiv, 2019; arXiv: 1905.11946 doi: 10.48550/arXiv.1905.11946.

Iandola F N, Han S, Moskewicz M W, Ashraf K, Dally W J, Keutzer K. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5 MB model size. arXiv, 2016; arXiv: 1602.07360. doi: 10.48550/arXiv.1602.07360.

Rosenfeld A, Thurston M. Edge and curve detection for visual scene analysis. IEEE Transactions on Computers, 1971; C-20(5): 562–569.

Panigrahi S, Nanda A, Swarnkar T. A survey on transfer learning. IEEE Transactions on Knowledge and Data Eengineering, 2010; 22(10): 1345–1359.

Shorten C, Khoshgoftaar T M. A survey on image data augmentation for deep learning. Journal of Big Data, 2019; 6(1): 60. doi: 10.1186/ s40537-019-0197-0.

Ren S Q, He K M, Girshick R, Sun J. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017; 39(6): 1137–1149.




Copyright (c) 2022 International Journal of Agricultural and Biological Engineering

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

2023-2026 Copyright IJABE Editing and Publishing Office