Natural UAV tele-operation for agricultural application by using Kinect sensor

Xuanchun Yin, Yubin Lan, Sheng Wen, Jiantao Zhang, Shifan Wu

Abstract


Remote-controlled (RC) unmanned aerial vehicles (UAVs) have been extensively applied in agricultural areas, such as remote sensing, precise spraying pesticides for crop protection, agricultural situation inspection and so on, but these telemanipulated UAVs systems are operated entirely by a ground-based pilot with a need of eyes focus on the remote site UAV flight. The key issue existed in agricultural UAV teleoperation area is a longtime training needed. In this paper, a novel natural UAV teleoperation control system in agricultural application was proposed. In UAV teleoperation scenario, human operator gestures measured by using Kinect sensor can be used as control commands for UAV flight. Moreover, some UAV teleoperation control commands related to human hand gestures were defined, which is similar to the radio gymnastic exercises in China. Therefore, gesture recognition-based UAV teleoperation control is easy to learn and master. In addition, a new real time human hand gesture recognition algorithm was proposed. The stability of UAV flight dynamic of roll, pitch and yaw as well as attitude control were verified with the experiments based on the proposed method. At last, the usability and effectiveness of the proposed method has been verified by the experimental results.
Keywords: gesture recognition, human pose estimation, depth image, skeleton tracking, teleoperation, unmanned aerial vehicle (UAV)
DOI: 10.25165/j.ijabe.20181104.4096

Citation: Yin X C, Lan Y B, Wen S, Zhang J T, Wu S F. Natural UAV tele-operation for agricultural application by using Kinect sensor. Int J Agric & Biol Eng, 2018; 11(4): 173-178.

Keywords


gesture recognition, human pose estimation, depth image, skeleton tracking, teleoperation, unmanned aerial vehicle (UAV)

Full Text:

PDF

References


Patel P. Agriculture drones are finally cleared for takeoff. IEEE Spectrum, 2016; 53(11): 13–14.

Lan Y B, Chen S D, Fritz B K. Current status and future trends of precision agricultural aviation technologies. Int J Agric & Biol Eng, 2017: 10(3): 1–17.

Chen S D, Lan Y B, Li J Y, Zhou Z Y, Liu A M, Mao Y D. Effect of wind field below unmanned helicopter on droplet deposition distribution of aerial spraying. Int J Agric & Biol Eng, 2017: 10(3): 67–77.

Li J Y, Lan Y B, Zhou Z Y, Zeng S, Huang C, Yao W X, et al. Design and test of operation parameters for rice air broadcasting by unmanned aerial vehicle. Int J Agric & Biol Eng, 2016: 9(5): 24–32.

Xue X, Lan Y B, Sun Z, Chang C, Hoffmann W C. Develop an unmanned aerial vehicle based automatic aerial spraying system. Computers and Electronics in Agriculture, 2016: 128: 58–66.

Zhang H, Han M, Chávez J L, Lan Y B. Improvement in estimation of soil water deficit by integrating airborne imagery data into a soil water balance model. Int J Agric & Biol Eng, 2017: 10(3): 37–46.

Li J Y, Lan Y B, Wang J W, Chen S D, Huang C, Liu Q, et al. Distribution law of rice pollen in the wind field of small UAV. Int J Agric & Biol Eng, 2017; 10(4): 32–40.

Costa F G, Ueyama J, Braun T, Pessin G, Osório F S, Vargas P A. The use of unmanned aerial vehicles and wireless sensor network in agricultural applications. International Conference on Geoscience and Remote Sensing Symposium (IGARSS), IEEE, July 2012; pp. 5045–5048.

Ahrens S, Levine D, Andrews G, How J P. Vision-based guidance and control of a hovering vehicle in unknown, GPS-denied environments. International Conference on Robotics and Automation, ICRA'09. IEEE, 2009; pp.2643–2648.

Chowdhary G, Johnson E N, Magree D, Wu A, Shein A. GPS‐denied indoor and outdoor monocular vision aided navigation and control of unmanned aircraft. Journal of Field Robotics, 2013: 30(3): 415–438.

Du G, Zhang P. Markerless human–robot interface for dual robot manipulators using Kinect sensor. Robotics and Computer-Integrated Manufacturing, 2014: 30(2): 150–159.

Han J, Shao L, Xu D, Shotton J. Enhanced computer vision with Microsoft Kinect sensor: A review. IEEE Trans. Cybern., 2013: 43: 1318–1334.

Roca D, Lagüela S, Díaz-Vilariño L, Armesto J, Arias P. Low-cost aerial unit for outdoor inspection of building façades. Automation in Construction, 2013: 36: 128–135.

Zhu G, Zhang L, Shen P, Song J. An online continuous human action recognition algorithm based on the Kinect sensor. Sensors, 2016: 16(2): 161.

Kim D, Kim D H, Kwak K C. Classification of K-Pop dance movements based on skeleton information obtained by a kinect sensor. Sensors, 2017: 17(6): 1261.

Sanna A, Lamberti F, Paravati G, Henao Ramirez E A, Manuri F. A Kinect-based natural interface for quadrotor control. Entertainment Computing, 2013: 4(3): 179–186.

Sun T, Nie S Y, Yeung D Y, Shen S J. Gesture-based piloting of an aerial robot using monocular vision. International Conference on Robotics and Automation (ICRA), IEEE, 2017; pp.5913–5920.

Peshkova E, Hitz M, Kaufmann B. Natural interaction techniques for an unmanned aerial vehicle system. IEEE Pervasive Computing, 2017; 1: 34-42.

Mettler B, Dadkhah N, Kong Z, Andersh J. Research infrastructure for interactive human-and autonomous guidance. Journal of Intelligent & Robotic Systems, 2013; 70(1-4): 437-459.

Fu C, Olivares-Mendez M A, Suarez-Fernandez R., Campoy P. Monocular visual-inertial slam-based collision avoidance strategy for fail-safe UAV using fuzzy logic controllers. Journal of Intelligent & Robotic Systems, 2014; 73(1-4): 513-533




Copyright (c) 2018



2023-2026 Copyright IJABE Editing and Publishing Office