Natural UAV tele-operation for agricultural application by using Kinect sensor

Authors

  • Xuanchun Yin 1. College of Engineering, South China Agricultural University, Guangzhou 510642, China;
  • Yubin Lan 1. College of Engineering, South China Agricultural University, Guangzhou 510642, China; 2. National Center for International Collaboration Research on Precision Agricultural Aviation Pesticides Spraying Technology, Guangzhou 510642, China; 3. International Laboratory of Agricultural Aviation Applied Technology, Guangzhou 510642, China;
  • Sheng Wen 4. Eengineering fundamental Teaching and Trainer Center, South China Agricultural University, Guangzhou 510642, China;
  • Jiantao Zhang 5. College of Mathematics and Informatics, South China Agricultural University, Guangzhou 510642, China
  • Shifan Wu 1. College of Engineering, South China Agricultural University, Guangzhou 510642, China;

DOI:

https://doi.org/10.25165/ijabe.v11i4.4096

Keywords:

gesture recognition, human pose estimation, depth image, skeleton tracking, teleoperation, unmanned aerial vehicle (UAV)

Abstract

Remote-controlled (RC) unmanned aerial vehicles (UAVs) have been extensively applied in agricultural areas, such as remote sensing, precise spraying pesticides for crop protection, agricultural situation inspection and so on, but these telemanipulated UAVs systems are operated entirely by a ground-based pilot with a need of eyes focus on the remote site UAV flight. The key issue existed in agricultural UAV teleoperation area is a longtime training needed. In this paper, a novel natural UAV teleoperation control system in agricultural application was proposed. In UAV teleoperation scenario, human operator gestures measured by using Kinect sensor can be used as control commands for UAV flight. Moreover, some UAV teleoperation control commands related to human hand gestures were defined, which is similar to the radio gymnastic exercises in China. Therefore, gesture recognition-based UAV teleoperation control is easy to learn and master. In addition, a new real time human hand gesture recognition algorithm was proposed. The stability of UAV flight dynamic of roll, pitch and yaw as well as attitude control were verified with the experiments based on the proposed method. At last, the usability and effectiveness of the proposed method has been verified by the experimental results. Keywords: gesture recognition, human pose estimation, depth image, skeleton tracking, teleoperation, unmanned aerial vehicle (UAV) DOI: 10.25165/j.ijabe.20181104.4096 Citation: Yin X C, Lan Y B, Wen S, Zhang J T, Wu S F. Natural UAV tele-operation for agricultural application by using Kinect sensor. Int J Agric & Biol Eng, 2018; 11(4): 173-178.

References

Patel P. Agriculture drones are finally cleared for takeoff. IEEE Spectrum, 2016; 53(11): 13–14.

Lan Y B, Chen S D, Fritz B K. Current status and future trends of precision agricultural aviation technologies. Int J Agric & Biol Eng, 2017: 10(3): 1–17.

Chen S D, Lan Y B, Li J Y, Zhou Z Y, Liu A M, Mao Y D. Effect of wind field below unmanned helicopter on droplet deposition distribution of aerial spraying. Int J Agric & Biol Eng, 2017: 10(3): 67–77.

Li J Y, Lan Y B, Zhou Z Y, Zeng S, Huang C, Yao W X, et al. Design and test of operation parameters for rice air broadcasting by unmanned aerial vehicle. Int J Agric & Biol Eng, 2016: 9(5): 24–32.

Xue X, Lan Y B, Sun Z, Chang C, Hoffmann W C. Develop an unmanned aerial vehicle based automatic aerial spraying system. Computers and Electronics in Agriculture, 2016: 128: 58–66.

Zhang H, Han M, Chávez J L, Lan Y B. Improvement in estimation of soil water deficit by integrating airborne imagery data into a soil water balance model. Int J Agric & Biol Eng, 2017: 10(3): 37–46.

Li J Y, Lan Y B, Wang J W, Chen S D, Huang C, Liu Q, et al. Distribution law of rice pollen in the wind field of small UAV. Int J Agric & Biol Eng, 2017; 10(4): 32–40.

Costa F G, Ueyama J, Braun T, Pessin G, Osório F S, Vargas P A. The use of unmanned aerial vehicles and wireless sensor network in agricultural applications. International Conference on Geoscience and Remote Sensing Symposium (IGARSS), IEEE, July 2012; pp. 5045–5048.

Ahrens S, Levine D, Andrews G, How J P. Vision-based guidance and control of a hovering vehicle in unknown, GPS-denied environments. International Conference on Robotics and Automation, ICRA'09. IEEE, 2009; pp.2643–2648.

Chowdhary G, Johnson E N, Magree D, Wu A, Shein A. GPS‐denied indoor and outdoor monocular vision aided navigation and control of unmanned aircraft. Journal of Field Robotics, 2013: 30(3): 415–438.

Du G, Zhang P. Markerless human–robot interface for dual robot manipulators using Kinect sensor. Robotics and Computer-Integrated Manufacturing, 2014: 30(2): 150–159.

Han J, Shao L, Xu D, Shotton J. Enhanced computer vision with Microsoft Kinect sensor: A review. IEEE Trans. Cybern., 2013: 43: 1318–1334.

Roca D, Lagüela S, Díaz-Vilariño L, Armesto J, Arias P. Low-cost aerial unit for outdoor inspection of building façades. Automation in Construction, 2013: 36: 128–135.

Zhu G, Zhang L, Shen P, Song J. An online continuous human action recognition algorithm based on the Kinect sensor. Sensors, 2016: 16(2): 161.

Kim D, Kim D H, Kwak K C. Classification of K-Pop dance movements based on skeleton information obtained by a kinect sensor. Sensors, 2017: 17(6): 1261.

Sanna A, Lamberti F, Paravati G, Henao Ramirez E A, Manuri F. A Kinect-based natural interface for quadrotor control. Entertainment Computing, 2013: 4(3): 179–186.

Sun T, Nie S Y, Yeung D Y, Shen S J. Gesture-based piloting of an aerial robot using monocular vision. International Conference on Robotics and Automation (ICRA), IEEE, 2017; pp.5913–5920.

Peshkova E, Hitz M, Kaufmann B. Natural interaction techniques for an unmanned aerial vehicle system. IEEE Pervasive Computing, 2017; 1: 34-42.

Mettler B, Dadkhah N, Kong Z, Andersh J. Research infrastructure for interactive human-and autonomous guidance. Journal of Intelligent & Robotic Systems, 2013; 70(1-4): 437-459.

Fu C, Olivares-Mendez M A, Suarez-Fernandez R., Campoy P. Monocular visual-inertial slam-based collision avoidance strategy for fail-safe UAV using fuzzy logic controllers. Journal of Intelligent & Robotic Systems, 2014; 73(1-4): 513-533

Downloads

Published

2018-08-08

How to Cite

Yin, X., Lan, Y., Wen, S., Zhang, J., & Wu, S. (2018). Natural UAV tele-operation for agricultural application by using Kinect sensor. International Journal of Agricultural and Biological Engineering, 11(4), 173–178. https://doi.org/10.25165/ijabe.v11i4.4096

Issue

Vol. 11 No. 4 (2018): IJABE

Section

Information Technology, Sensors and Control Systems

License

IJABE is an international peer reviewed open access journal, adopting Creative Commons Copyright Notices as follows.


Authors who publish with this journal agree to the following terms:


  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.

  2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.

  3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).