Review of the field environmental sensing methods based on multi-sensor information fusion technology

Yuanyuan Zhang, Bin Zhang, Cheng Shen, Haolu Liu, Jicheng Huang, Kunpeng Tian, Zhong Tang

Abstract


Field environmental sensing can acquire real-time environmental information, which will be applied to field operation, through the fusion of multiple sensors. Multi-sensor fusion refers to the fusion of information obtained from multiple sensors using more advanced data processing methods. The main objective of applying this technology in field environment perception is to acquire real-time environmental information, making agricultural mechanical devices operate better in complex farmland environment with stronger sensing ability and operational accuracy. In this paper, the characteristics of sensors are studied to clarify the advantages and existing problems of each type of sensors and point out that multiple sensors can be introduced to compensate for the information loss. Secondly, the mainstream information fusion types at present are outlined. The characteristics, advantages and disadvantages of different fusion methods are analyzed. The important studies and applications related to multi-sensor information fusion technology published at home and abroad are listed. Eventually, the existing problems in the field environment sensing at present are summarized and the prospect for future of sensors precise sensing, multi-dimensional fusion strategies, discrepancies in sensor fusion and agricultural information processing are proposed in hope of providing reference for the deeper development of smart agriculture.
Key words: multi-sensor; information fusion; field environmental sensing; fusion methods; smart agriculture
DOI: 10.25165/j.ijabe.20241702.8596

Citation: Zhang Y Y, Zhang B, Shen C, Liu H L, Huang J C, Tian K P, et al. Review of the field environmental sensing methods based on multi-sensor information fusion technology. Int J Agric & Biol Eng, 2024; 17(2): 1–13.

Keywords


multi-sensor; information fusion; field environmental sensing; fusion methods; smart agriculture

Full Text:

PDF

References


Xian J B. A research of ES-based multi-sensor data fusion technology and its application on mobile-robot obstacle avoidance. North China University of Technology, 2008. (in Chinese)

Xu B W, Ma Z Y, Li Y. Research progress and application of multi-sensor information fusion technology in environmental perception. Computer Measurement & Control, 2022; 30(9): 1–7, 21.

Kwon Y D, Lee J S. A stochastic map building method for mobile robot using 2-D laser range finder. Autonomous Robots, 1999; 7(2): 187–200.

Zhang K L, Hu Y, Yang L, Zhang D X, Cui T, Fan L L. Design and experiment of auto-follow row system for corn harvester. Transactions of the CSAM, 2020; 51(2): 103–14. (in Chinese)

Shang Y H, Wang H, Meng Z J, Yin Y X, Xiao Y J, Song Z H. Rice and wheat harvesting boundary detection and automatic alignment system based on LiDAR. Transactions of the CSAM, 2023; 54(5): 19–28, 46.

Shang Y H, Zhang G Q, Meng Z J, Wang H, Su C H, Song Z H. Field obstacle detection method of 3D LiDAR point cloud based on Eucildean clustering. Transactions of the CSAM, 2022; 53(1): 23–32. (in Chinese)

Yang L L, Xu Y Y, Li Y B, Chang M S, Chen Z B, Lan Y B, et al. Real-Time field road freespace extraction for agricultural machinery autonomous driving based on LiDAR. Computers and Electronics in Agriculture, 2023; 211: 108028.

Liu M C. Research on inspection and obstacle avoidance methods of agro-machinery obstructions. Northwest A&F University, 2018. (in Chinese)

Ji Y H, Li S C, Peng C, Xu H Z, Cao R Y, Zhang M. Obstacle detection and recognition in farmland based on fusion point cloud data. Computers and Electronics in Agriculture, 2021; 189: 106409.

Wang Z G, Zhan J, Duan C G, Guan X, Lu P P, Yang K. A review of vehicle detection techniques for intelligent vehicles. IEEE Transactions on Neural Networks and Learning Systems, 2023; 34(8): 3811–31.

Tan Z. Application of millimeter wave in obstacle avoidance of plant protection UAV. Fuyang Normal University, 2021. (in Chinese)

Xue J L, Cheng F, Wang B Q, Li Y Q, Ma Z B, Chu Y Y. Method for millimeter wava radar farm obstacle detection based on invalid target filtering. Transactions of the CSAM, 2023; 54(4): 233–40. (in Chinese)

Henry D, Aubert H, Galaup P, Véronèse T. Dynamic estimation of the yield in precision viticulture from mobile millimeter-wave radar systems. IEEE Transactions on Geoscience and Remote Sensing, 2022; 60: 4704915.

Wang S B, Song J L, Qi P, Yuan C J, Wu H C, Zhang L T, et al. Design and development of orchard autonomous navigation spray system. Frontiers in Plant Science, 2022; 13.

Zhang Y, Pan S Q, Xie Y S, Chen K, Mo J Q. Detection of ridge in front of agricultural machinery by fusion of camera and millimeter wave radar. Transactions of the CSAE, 2021; 37(15): 169–178. (in Chinese)

He Y, Jiang H, Fang H, Wang Y, Liu Y F. Research progress of intelligent obstacle detection methods of vehicles and their application on agriculture. Transactions of the CSAE, 2018; 34(9): 21–32. (in Chinese)

Vinod D N, Singh T. Autonomous farming and surveillance agribot in adjacent boundary; proceedings of the 9th International Conference on Computing, Communication and Networking Technologies (ICCCNT), IISC, Bengaluru, IEEE, India, Jul 10-12, 2018.

Li X H, Yan S, Gao N N, An X F, Wu G W, Meng Z J. Accurate evaluation system for wheat topdressing based on ultrasonic sensor. Transactions of the CSAM, 2020; 51(S1): 203–209. (in Chinese)

Hao S K, Huo J Q, Zhang Y, Li Z W. Self-propelled control system between rows of agricultural machinery based on ultrasonic distance measurement. Electronic Design Engineering, 2022; 30(21): 48–55. (in Chinese)

Borenstein J, Koren Y. Real-time obstacle avoidance for fast mobile robots. IEEE Transactions on Systems Man and Cybernetics, 1989; 19(5): 1179–87.

Li J, Xu Y, Jiang R, Yang Z, Lu H Z. Establishment and verification of model for ultrasonic soil water content detector. Transactions of the CSAE, 2017; 33(13): 127–133. (in Chinese)

Fu G P, Yang C Y, Zhang S A, Huang W F, Chen T C, Zhu L X. Research on laser and ultrasonic combined ranging method for robot navigation at banana plantation. Transactions of the CSAM, 2021; 52(5): 159–68. (in Chinese)

Hu J W, Zheng B Y, Wang C, Zhao C H, Hou X L, Pan Q, et al. A survey on multi-sensor fusion based obstacle detection for intelligent ground vehicles in off-road environments. Frontiers of Information Technology & Electronic Engineering, 2020; 21(5): 675–692.

Wang Q, Liu H, Yang P S, Meng Z J. Detection method of headland boundary line based on machine vision. Transactions of the CSAM, 2020; 51(5): 18–27. (in Chinese)

Li Y, Huang D Y, Qi J T, Chen S K, Sun H B, Liu H L, et al. Feature point registration model of farmland surface and its application based on a monocular camera. Sensors, 2020; 20(13).

Cadena C, Carlone L, Carrillo H, Latif Y, Scaramuzza D, Neira J, et al. Past, present, and future of simultaneous localization and mapping: toward the robust-perception age. IEEE Transactions on Robotics, 2016; 32(6): 1309–1332.

Wang Z H, Cai Y F, Wang H, Chen L, Xiong X X. Surrounding multi-target trajectory prediction method based on monocular visual motion estimation. Automotive Engineering, 2022; 44(9): 1318–26, 71. (in Chinese)

Qi Y S, Chen P L, Liu L Q, Dong C Y. Simultaneous localization and multi-mapping algorithm in dynamic environment based on monocular vision. Transactions of the CSAM, 2022; 53(4): 280–292. (in Chinese)

Cheng J Y. Research on moving obstacle detection and avoidance strategy for agricultural robot based on machine vision. Nanjing Agricultural University, 2011. (in Chinese)

Hong Z J, Li Y M, Lin H Z, Gong L, Liu C L. Field boundary distance detection method in early stage of planting based on stereo vision. Transactions of the CSAM, 2022; 53(5): 27–33, 56. (in Chinese)

Wei X H, Zhang M, Liu Q S, Li L. Extraction of crop height and cut-edge information based on stereo vision. Transactions of the CSAM, 2022; 53(3): 225–233. (in Chinese)

Reina G, Milella A. Towards Autonomous Agriculture: Automatic Ground Detection Using Trinocular Stereovision. Sensors, 2012; 12(9): 12405–12423.

Wei J, Rovira-mas F, Reid J F, Han S. Obstacle detection using stereo vision to enhance safety of autonomous machines. Transactions of the Asae, 2005; 48(6): 2389–97.

Kragh M, Underwood J. Multimodal obstacle detection in unstructured environments with conditional random fields. Journal of Field Robotics, 2020; 37(1): 53–72.

Cai D Q. Research on autonomous operation perception technology in unstructured. Shanghai JiaoTong University, 2020. (in Chinese)

Lahat D, Adali T, Jutten C. Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects. Proceedings of the IEEE, 2015; 103(9): 1449–1477.

Silva D V, Roche J, Kondoz A. Robust fusion of LiDAR and wide-angle camera data for autonomous mobile robots. Sensors, 2018; 18(8): 2730.

Sun Z D. Multi-source data fusion oriented Bayesian estimation method research. Journal of Qilu University of Technology, 2018; 32(1): 73–76. (in Chinese)

Sock J, Kim J, Min J H, Kwak K. Probabilistic traversability map generation using 3D-LIDAR and camera. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Royal Inst Technol, Ctr Autonomous Syst, Stockholm, Sweden, May 16-21, 2016.

Zhao K Y, Sun R T, Li L, Hou M M, Yuan G, Sun R Z. An improved evidence fusion algorithm in multi-sensor systems. Applied Intelligence, 2021; 51(11): 7614–7624.

Wang S, Ren Y, Guan X Z, Wang J. Multi-source data fusion method based on difference information. Journal of Northeastern University (Natural Science), 2021; 42(9): 1246–53. (in Chinese)

Yan H C, Huang X H, Wang M. Multi-sensor data fusion technique and its application. Journal of Transducer Technology, 2005; 24(10): 6–9. (in Chinese)

Tan B C, Li B. The weighted average of the data fusion algorithm research of driverless vehicle sensor system. Electronic Design Engineering, 2015; 23(16): 95–97. (in Chinese)

Zhen M, Wang S P. An adaptive weighted average fusion method for visible and infrared images. Infrared Technology, 2019; 41(4): 341–346. (in Chinese)

Meng Z H, Ho Q H, Huang Z F, Guo H L, Ang Jr M H, Rus D. Online multi-target tracking for maneuvering vehicles in dynamic road context. Computer Science, 2019; arXiv: 1912.00603doi: 10.48550/arXiv.1912.00603.

Liu Q H, Zhou W Q, Zhang Y K, Fei X. Multi-target detection based on multi-sensor redundancy and dynamic weight distribution for driverless cars. IEEE 3rd International Conference on Communication, Information System and Computer Engineering (CISCE), 2021; pp.229–234.

Peng W Z, Ao Y H, Huang X T, Wang P F. Automatic vehicle location and state estimation based on multi-senor data fusion. Journal of Transduction Technology, 2020; 33(8): 1140–8. (in Chinese)

Hao X J, Li G X, Li M Z, Zhang Y F, Chang X F. Research of UKF in the target tracking. Electronic Design Engineering, 2012; 20(13): 161–164. (in Chinese)

Wu L, Lu F X, Liu Z. UKF algorithm and its applications to passive target tracking. Systems Engineering and Electronics, 2005; 27(1): 49–51, 75. (in Chinese)

Qi Y J, Wang Q. Review of multi-source data fusion algorithm. Aerospace Electronic Warfare, 2017; 33(6): 37–41. (in Chinese)

Wei Z, Ye J H, Shen S Z. Engineering application of the maximum entropy reliability theory. Journal of Vibration and Shock, 2007; 26(6): 146–148, 51, 90. (in Chinese)

Magalhaess J, Ruger S. An information-theoretic framework for semantic-multimedia retrieval. ACM Transactions on Information Systems, 2010; 28(4): 1–32.

Yang Y X. Researching multi-sensor information fusion using D-S theory and fuzzy set theory. Lanzhou University of Technology, 2011. (in Chinese)

Liu Z W, Zhao S S, Yang B, Yi M J. Clustering method to discriminate active false targets in multistatic radar system. Journal of Electronics & Information Technology, 2021; 43(11): 3211–3219. (in Chinese)

Liu M, Rong X W, Li Y B, Zhang S S, Yin Y F, Ruan J H. Speed adaptive control of mobile robot based on terrain clustering analysis. Journal of Jilin University (Engineering and Technology Edition), 2021; 51(4): 1496–505. (in Chinese)

Lu W, Zeng M J, Qin H H. Intelligent navigation algorithm of plant phenotype detection robot based on dynamic credibility evaluation. Int J Agric & Biol Eng, 2021; 14(6): 195–206.

Bai J, Li S, Huang L B. Robust detection and tracking method for moving object based on radar and camera data fusion. IEEE Sensors Journal, 2021; 21(9): 10761–10774.

Weber D, Guhmann C, Seeel T. Neural networks versus conventional filters for inertial-sensor-based attitude estimation. Proceedings of the 23rd International Conference on Information Fusion (FUSION), Electr Network, Jul 06-09, 2020.

Li J. Research on target recognition algorithm of intelligent learning. Infrared (Monthly), 2003; 2: 11–17, 32. (in Chinese)

Niu L H, Ni G Q. Feature optimization for multi-sensor target recognition system. Optical Technique, 2005; 31(3): 420–423, 6. (in Chinese)

Liu J G, Huang J, Sun R, Yu H T, Xiao R D. Data fusion for multi-source sensors using GA-PSO-BP neural network. IEEE Transactions on Intelligent Transportation Systems, 2021; 22(10): 6583–98.

Wang H X, Wu Q F, Wu X B, Zhang Q Y. Application research of multi-sensor fusion in the robot position perception. Mechanical & Electrical Engineering Technology, 2020; 49(12): 89–91. (in Chinese)

Yang J H. Pose tracking and path planning for UAV based on multi-sensor fusion. Zhejiang University, 2019. (in Chinese)

Zhen X Y, Lai J Z, Lv P, Yuan C, Fan W S. Object detection and positioning method based on infrared vision/lidar fusion. Navigating Positioning & Timing, 2021; 8(3): 34–41. (in Chinese)

Huang Y R, Fu J H, Xu S Y, Han T, Liu Y W. Research on integrated navigation system of agricultural machinery based on RTK-BDS/INS. Agriculture-Basel, 2022; 12(8): 1169.

Tang J P, Song H S, Wang D S. Research of path planning of autonomous robot in dynamic environment. Journal of Zhengzhou University (Engineering Science), 2012; 44(1): 75–78. (in Chinese)

Lv P F, Wang B Q, Cheng F, Xue J L. Multi-objective association detection of farmland obstacles based on information fusion of millimeter wave radar and camera. Sensors, 2023; 23(1): 230.

Yang L, Chen F X, Chen K Y, Liu S N. Research and application of obstacle avoidance method based on multi-sensor for UAV. Computer Measurement & Control, 2019; 27(1): 280–283, 7. (in Chinese)

He J, He J, Luo X W, Li W C, Man Z X, Feng D W. Rice row recognition and navigation control based on multi-sensor fusion. Transactions of the CSAM, 2022; 53(3): 18–26, 137. (in Chinese)

Wang G, Huang D Y, Zhou D Y, Liu H L, Qu M H, Ma Z Y. Maize (Zea mays L. ) seedling detection based on the fusion of a modified deep learning model and a novel Lidar points projecting strategy. Int J Agric & Biol Eng, 2022; 15(5): 172–180.

Zhang M, Ji Y F, Li S C, Cao R Y, Xu H Z, Zhang Z Q. Research progress of agricultural machinery navigation technology. Transactions of the CSAM, 2020; 51(4): 1–18. (in Chinese)

Zhou J, He Y Q. Research progress on navigation path planning of agricultural machinery. Transactions of the CSAM, 2021; 52(9): 1–14. (in Chinese)

Li A J, Cao J P, Li S M, Huang Z, Wang J B, Liu G. Map construction and path planning method for a mobile robot based on multi-sensor information fusion. Applied Sciences-Basel, 2022; 12(6).

Jiang L T, Chi R J, Xiong Z X, Ma Y Q, Ban C, Zhu X L. Obstacle winding strategy of rice transplanter based on optimized artificial potential field method. Transactions of the CSAM, 2022; 53(S1): 20–27. (in Chinese)




Copyright (c) 2024 International Journal of Agricultural and Biological Engineering

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

2023-2026 Copyright IJABE Editing and Publishing Office