Three-dimensional reconstruction and phenotypic identification of the wheat plant using RealSense D455 sensor

Authors

  • Ming Li Shandong Engineering Research Center of Agricultural Equipment Intelligentization, Shandong Key Laboratory of Intelligent Production Technology and Equipment for Facility Horticulture, College of Mechanical and Electronic Engineering, Shandong Agricultural University, Tai’an 271018, Shandong, China
  • Wanteng Zhang Shandong Engineering Research Center of Agricultural Equipment Intelligentization, Shandong Key Laboratory of Intelligent Production Technology and Equipment for Facility Horticulture, College of Mechanical and Electronic Engineering, Shandong Agricultural University, Tai’an 271018, Shandong, China
  • Weiting Pan Shandong Engineering Research Center of Agricultural Equipment Intelligentization, Shandong Key Laboratory of Intelligent Production Technology and Equipment for Facility Horticulture, College of Mechanical and Electronic Engineering, Shandong Agricultural University, Tai’an 271018, Shandong, China
  • Junke Zhu School of Agricultural Engineering and Food Science, Shandong University of Technology, Zibo 255100, Shandong, China
  • Xubin Song Shandong Engineering Research Center of Agricultural Equipment Intelligentization, Shandong Key Laboratory of Intelligent Production Technology and Equipment for Facility Horticulture, College of Mechanical and Electronic Engineering, Shandong Agricultural University, Tai’an 271018, Shandong, China;
  • Chunying Wang 1. Shandong Engineering Research Center of Agricultural Equipment Intelligentization, Shandong Key Laboratory of Intelligent Production Technology and Equipment for Facility Horticulture, College of Mechanical and Electronic Engineering, Shandong Agricultural University, Tai’an 271018, Shandong, China; 3. State Key Laboratory of Wheat Improvement, Shandong Agricultural University, Tai’an 271018, Shandong, China
  • Ping Liu 1. Shandong Engineering Research Center of Agricultural Equipment Intelligentization, Shandong Key Laboratory of Intelligent Production Technology and Equipment for Facility Horticulture, College of Mechanical and Electronic Engineering, Shandong Agricultural University, Tai’an 271018, Shandong, China; 3. State Key Laboratory of Wheat Improvement, Shandong Agricultural University, Tai’an 271018, Shandong, China

DOI:

https://doi.org/10.25165/ijabe.v18i4.9169

Keywords:

wheat plant, RealSense sensor, MVS, three-dimensional point cloud, phenotypic traits

Abstract

Accurate and rapid wheat morphology reconstruction and trait collection are essential for selecting varieties, scientific cultivation, and precise management. A single perspective is limited by environmental obstructions, hindering the collection of high-throughput phenotype data for wheat plants. Therefore, a rapid reconstruction method of multi-view three dimensional point cloud is proposed to realize the high-throughput and accurate identification of wheat phenotype. Firstly, taking wheat at the tillering stage as the experimental object, a multi-view acquisition system based on a RealSense sensor was constructed, and the point cloud data of wheat were obtained from 16 views. Secondly, a joint photometric and geometric objective was optimized, and space location was registered by colored Point Cloud Registration (colored) and Iterative Closest Point (ICP) algorithms. Furthermore, the Multiple View Stereo (MVS) algorithm was used to combine the depth image, RGB image, and spatial position obtained by coarse registration to enable the fine registration of multi-viewpoint clouds. Compared with the traditional Structure From Motion (SFM)-MVS algorithm, our proposed method is much faster, with an average reconstruction time of 33.82 s. Moreover, the wheat plant height, leaf length, leaf width, leaf area, and leaf angle of wheat were calculated based on the three-dimensional point cloud of the wheat plant. The experimental results showed that the determination coefficients of the method are 0.996, 0.958, 0.956, 0.984, and 0.849, respectively. Finally, phenotypic information such as compact degree, convex hull volume, and average leaf area of different wheat varieties were analyzed and identified, proving that the method could capture the phenotypic differences between varieties and individuals. The proposed method provides a rapid approach to quantify wheat phenotypic traits, aiding breeding, scientific cultivation, and environmental management. Keywords: wheat plant, RealSense sensor, MVS, three-dimensional point cloud, phenotypic traits DOI: 10.25165/j.ijabe.20251804.9169 Citation: Li M, Zhang W T, Pan W T, Zhu J K, Song X B, Wang C Y, et al. Three-dimensional reconstruction and phenotypic identification of the wheat plant using RealSense D455 sensor. Int J Agric & Biol Eng, 2025; 18(4): 254–265.

References

Kaushal S, Gill H S, Billah M M, Khan S N, Halder J, Bernardo A, et al. Enhancing the potential of phenomic and genomic prediction in winter wheat breeding using high-throughput phenotyping and deep learning. Frontiers in Plant Science, 2024; 15: 1410249.

Ren A H, Jiang D, Kang M, Wu J, Xiao F C, Hou P, et al. Evaluation of an intelligent artificial climate chamber for high-throughput crop phenotyping in wheat. Plant Methods, 2022; 18: 1–15.

Spanic V, Lalic Z, Berakovic I, Jukic G, Varnica I. Morphological characterization of 1322 winter wheat (Triticum aestivum L.) varieties from EU referent collection. Agriculture, 2024; 14: 551.

Watanabe K, Guo W, Arai K, Takanashi H, Kajiya-Kanegae H, Kobayashi M, et al. High-Throughput phenotyping of sorghum plant height using an unmanned aerial vehicle and its application to genomic prediction modeling. Front Plant Sci, 2017; 8.

Derbyshire M C, Batley J, Edwards D. Use of multiple ‘omics techniques to accelerate the breeding of abiotic stress tolerant crops. Current Plant Biology, 2022; 32: 100262.

Xiao Q, Bai X, Zhang C, He Y. Advanced high-throughput plant phenotyping techniques for genome-wide association studies: A review. Journal of Advanced Research, 2022; 35: 215–30.

Wang Y H, Su W H. Convolutional neural networks in computer vision for grain crop phenotyping: A review. Agronomy, 2022; 12: 2659.

Sun D, Robbins K, Morales N, Shu Q, Cen H. Advances in optical phenotyping of cereal crops. Trends in Plant Science 2022; 27: 191–208.

Weyler J, Magistri F, Seitz P, Behley J, Stachniss C. In-Field Phenotyping Based on Crop Leaf and Plant Instance Segmentation, 2022, p. 2725–34.

Tian Y, Zhang J, Zhang Z, Wu J. Research on super-resolution enhancement technology using improved transformer network and 3D reconstruction of wheat grains. IEEE Access 2024; 12: 62882–62898.

Li Y, Wen W, Miao T, Wu S, Yu Z, Wang X, et al. Automatic organ-level point cloud segmentation of maize shoots by integrating high-throughput data acquisition and deep learning. Computers and Electronics in Agriculture, 2022; 193: 106702.

Kargar R A, MacKenzie R, Asner G P, van Aardt J. A density-based approach for leaf area index assessment in a complex forest environment using a terrestrial laser scanner. Remote Sensing, 2019; 11: 1791.

Sun S, Li C, Paterson A H, Jiang Y, Xu R, Robertson J S, et al. In-field high throughput phenotyping and cotton plant growth analysis using LiDAR. Front Plant Sci, 2018; 9.

Zhu R S, Li S, Sun Y Z, Cao Y Y, Sun K, Guo Y X, et al. Research Advances and Prospects of Crop 3D Reconstruction Technology. Smart Agriculture, 2021; 3: 94.

Forero M G, Murcia H F, Méndez D, Betancourt-Lozano J. LiDAR platform for acquisition of 3D plant phenotyping database. Plants, 2022; 11: 2199.

Chen G, Hu L, Luo X, Wang P, He J, Huang P, et al. A review of global precision land-leveling technologies and implements: Current status, challenges and future trends. Computers and Electronics in Agriculture, 2024; 220: 108901.

Pound M P, French A P, Murchie E H, Pridmore T P. Automated recovery of three-dimensional models of plant shoots from multiple color images. Plant Physiology, 2014; 166: 1688–98.

Westoby M J, Brasington J, Glasser N F, Hambrey M J, Reynolds J M. ‘Structure-from-Motion’ photogrammetry: A low-cost, effective tool for geoscience applications. Geomorphology 2012; 179: 300–314.

Furukawa Y, Hernández C. Multi-view stereo: A Tutorial. CGV, 2015; 9: 1–148.

Wu S, Wen W L, Wang Y J, Fan J C, Wang C Y, Gou W B, et al. MVSPheno: A portable and low-cost phenotyping platform for maize shoots using multiview stereo 3D reconstruction. Plant Phenomics, 2020; 2020: 1848437.

Wu S, Wen W, Gou W, Lu X, Zhang W, Zheng C, et al. A miniaturized phenotyping platform for individual plants using multi-view stereo 3D reconstruction. Frontiers in Plant Science, 2022; 13.

Janiszewski M, Prittinen M, Torkan M, Uotinen L. Rapid tunnel scanning using a 360-degree camera and SfM photogrammetry. IOP Conf Ser: Earth Environ Sci, 2023; 1124: 012010.

Ye Z, Bao C, Zhou X, Liu H, Bao H, Zhang G. EC-SfM: Efficient Covisibility-based Structure-from-Motion for both sequential and unordered images. arXivOrg, 2023.

Kang H, Wang X, Chen C. Accurate fruit localisation using high resolution LiDAR-camera fusion and instance segmentation. Computers and Electronics in Agriculture, 2022; 203: 107450.

Wu F, Wang J, Zhou Y, Song X, Ju C, Sun C, et al. Estimation of winter wheat tiller number based on optimization of gradient vegetation characteristics. Remote Sensing, 2022; 14: 1338.

Tadic V, Toth A, Vizvari Z, Klincsik M, Sari Z, Sarcevic P, et al. Perspectives of realsense and ZED depth sensors for robotic vision applications. Machines, 2022; 10: 183.

Vandendaele B, Martin-Ducup O, Fournier R A, Pelletier G, Lejeune P. Mobile laser scanning for estimating tree structural attributes in a temperate hardwood forest. Remote Sensing, 2022; 14: 4522.

Zou R, Zhang Y, Chen J, Li J, Dai W, Mu S. Density estimation method of mature wheat based on point cloud segmentation and clustering. Computers and Electronics in Agriculture 2023; 205: 107626.

Liu R, Chen D, Liu T, Xiong Z, Yuan Z. Learning to predict 3D lane shape and camera pose from a single image via geometry constraints. Proceedings of the AAAI Conference on Artificial Intelligence, 2022; 36: 1765–1772.

Vázquez-Arellano M, Paraforos D S, Reiser D, Garrido-Izard M, Griepentrog H W. Determination of stem position and height of reconstructed maize plants using a time-of-flight camera. Computers and Electronics in Agriculture, 2018; 154: 276–88.

Zhu K, Ma X, Guan H, Feng J, Zhang Z, Yu S. A method of calculating the leafstalk angle of the soybean canopy based on 3D point clouds. International Journal of Remote Sensing, 2021.

Fan Z, Sun N, Qiu Q, Li T, Feng Q, Zhao C. In situ measuring stem diameters of maize crops with a high-throughput phenotyping robot. Remote Sensing, 2022; 14: 1030.

Nair N S, Nair M S. Multi-view stereo using graph cuts-based depth refinement. IEEE Signal Processing Letters, 2022; 29: 1903–1907.

Zine-El-Abidine M, Dutagaci H, Galopin G, Rousseau D. Assigning apples to individual trees in dense orchards using 3D colour point clouds. Biosystems Engineering, 2021; 209: 30–52.

Chen Y, Xiong Y, Zhang B, Zhou J, Zhang Q. 3D point cloud semantic segmentation toward large-scale unstructured agricultural scene classification. Computers and Electronics in Agriculture 2021; 190: 106445.

Jin S, Sun X, Wu F, Su Y, Li Y, Song S, et al. Lidar sheds new light on plant phenomics for plant breeding and management: Recent advances and future prospects. ISPRS Journal of Photogrammetry and Remote Sensing, 2021; 171: 202–23.

Ruchay A, Dorofeev K, Kober A. An efficient detection of local features in depth maps. Applications of Digital Image Processing XLI, vol. 10752, SPIE; 2018; pp.647–654.

Yao Z, Zhao Q, Li X, Bi Q. Point cloud registration algorithm based on curvature feature similarity. Measurement 2021; 177: 109274.

Liang P, Lin W, Luo G, Zhang C. Research of hand–eye system with 3D vision towards flexible assembly application. Electronics, 2022; 11: 354.

Downloads

Published

2025-08-21

How to Cite

Li, M., Zhang, W., Pan, W., Zhu, J., Song, X., Wang, C., & Liu, P. (2025). Three-dimensional reconstruction and phenotypic identification of the wheat plant using RealSense D455 sensor. International Journal of Agricultural and Biological Engineering, 18(4), 254–265. https://doi.org/10.25165/ijabe.v18i4.9169

Issue

Vol. 18 No. 4 (2025): IJABE

Section

Information Technology, Sensors and Control Systems

License

IJABE is an international peer reviewed open access journal, adopting Creative Commons Copyright Notices as follows.


Authors who publish with this journal agree to the following terms:


  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.

  2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.

  3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).