Deep learning for smart agriculture: Concepts, tools, applications, and opportunities

Nanyang Zhu, Xu Liu, Ziqian Liu, Kai Hu, Yingkuan Wang, Jinglu Tan, Min Huang, Qibing Zhu, Xunsheng Ji, Yongnian Jiang, Ya Guo

Abstract


In recent years, Deep Learning (DL), such as the algorithms of Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN) and Generative Adversarial Networks (GAN), has been widely studied and applied in various fields including agriculture. Researchers in the fields of agriculture often use software frameworks without sufficiently examining the ideas and mechanisms of a technique. This article provides a concise summary of major DL algorithms, including concepts, limitations, implementation, training processes, and example codes, to help researchers in agriculture to gain a holistic picture of major DL techniques quickly. Research on DL applications in agriculture is summarized and analyzed, and future opportunities are discussed in this paper, which is expected to help researchers in agriculture to better understand DL algorithms and learn major DL techniques quickly, and further to facilitate data analysis, enhance related research in agriculture, and thus promote DL applications effectively.
Keywords: deep learning, smart agriculture, neural network, convolutional neural networks, recurrent neural networks, generative adversarial networks, artificial intelligence, image processing, pattern recognition
DOI: 10.25165/j.ijabe.20181104.4475

Citation: Zhu N Y, Liu X, Liu Z Q, Hu K, Wang Y K, Tan J L, et al. Deep learning for smart agriculture: Concepts, tools, applications, and opportunities. Int J Agric & Biol Eng, 2018; 11(4): 32-44.

Keywords


deep learning, smart agriculture, neural network, convolutional neural networks, recurrent neural networks, generative adversarial networks, artificial intelligence, image processing, pattern recognition

Full Text:

PDF

References


Schalkoff R J. Artificial Neural Networks. Vol. 1. New York: McGraw-Hill, 1997.

Schmidhuber J. Deep learning in neural networks: An overview. Neural Networks, 2015; 61: 85–117.

Kamilaris A, Prenafeta-Boldú F X. Deep learning in agriculture: A survey. Computers and Electronics in Agriculture, 2018; 147: 70–90.

Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. Cornell University Library, Available at: https://arxiv.org/abs/1409.1556.

Simonyan K, Zisserman A. Two-stream convolutional networks for action recognition in videos. Advances in Neural Information Processing Systems, 2014; 1-4: 568–576.

Fan Y, Qian Y, Xie F, Soong F K. TTS synthesis with bidirectional LSTM based recurrent neural networks. Proc. Interspeech, 2014; pp.1964–1968.

Alex G, Schmidhuber J. Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Networks, 2005; 18(5-6): 602–610.

Alec R, Metz L, Chintala S. Unsupervised representation learning with deep convolutional generative adversarial networks. Cornell University Library, Available at: https://arxiv.org/abs/1511.06434.

Suárez P L, Sappa A D, Vintimilla B X. Infrared image colorization based on a triplet dcgan architecture. IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2017.

Jin J Q, Fu K, Zhang C H. Traffic sign recognition with hinge loss trained convolutional neural networks. IEEE Transactions on Intelligent Transportation Systems, 2014; 15(5): 1991–2000.

Haykin S S. Neural Networks and Learning Machines. Pearson Schweiz Ag, 2008.

Alpaydin E. Neural Networks and Deep Learning, 2016; 224p.

Hecht-Nielsen R C. Neural Networks for Perception. Academic Press, Inc., 1992; pp.65-93. Available at: https://doi.org/10.1016/B978-0-12- 741252-8.50002-9

Goh A T C. Back-propagation neural networks for modeling complex systems. Artificial Intelligence in Engineering, 1995; 9(3): 143–151.

Li J C, Wei H G. BP neural network used in recognition algorithm for star pattern. AOPC 2017: Optical Sensing and Imaging Technology and Applications. Vol. 10462. International Society for Optics and Photonics, 2017.

Xia X L, Han R Y. Research on classifications of mobile networking device users based on BP neural networks. 2nd International Conference on Mechatronics and Information Technology (ICMIT 2017), 2017; pp.104–109.

Razavian A S, Azizpour H, Sullivan J, Carlsson S. CNN features off-the-shelf: an astounding baseline for recognition. IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2014.

Vincent P, Larochelle H, Bengio Y, Manzagol P A. Extracting andcomposing robust features with denoising autoencoders. In Proc. 25th Int.Conf. on Machine Learning, 2008; pp.1096–1103.

LeCun Y, Bengio Y, Hinton G. Deep learning. Nature, 2015; 521(7553): 436.

Bouvrie J. Notes on convolutional neural networks. Cambridge: MIT, 2006.

Yin W P, Kann K, Yu M, Schütze H. Comparative study of CNN and RNN for natural language processing. Cornell University Library, 2017. Available at: https://arxiv.org/abs/1702.01923.

Parkhi O M, Vedaldi A, Zisserman A. Deep Face Recognition. BMVC, 2015; 1(3): 1–12.

Chung J Y, Gulchre C, Cho K, Bengio Y. Gated feedback recurrent neural networks. International Conference on Machine Learning. Lille, France, 2015.

Sak H, Senior A W. Processing acoustic sequences using long short-term memory (LSTM) neural networks that include recurrent projection layers. U.S. Patent No. 9,620,108. 11 Apr. 2017.

Han S, Kang J L, Mao H Z, Hu Y M, Li X, Li Y B, et al. Ese: Efficient speech recognition engine with sparse LSTM on FPGA. Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays. ACM, 2017.

Britz D, Goldie A, Luong M T, Le Q. Massive exploration of neural machine translation architectures. ACL, 2017. Available at: https://arxiv.org/pdf/1703.03906.pdf.

Radford A, Metz L, Chintala S. Unsupervised representation learning with deep convolutional generative adversarial networks. ICLR, 2016. Available at: https://arxiv.org/abs/1511.06434.

Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, et al. Generative adversarial nets. Proceedings of Advances in Neural Information Processing Systems, 2014.

Schoner G, Kelso J A. Dynamic pattern generation in behavioral and neural systems. Science, 1988; 239(4847): 1513–1520.

Denton E L, Chintala S, Fergus R. Deep generative image models using a laplacian pyramid of adversarial networks. Proceedings of Advances in Neural Information Processing Systems, 2015.

Ledig C, Theis L, Huszar F, Caballero J, Cunningham A, Acosta A, et al. Photo-realistic single image super-resolution using a generative adversarial network. In Proceeding of IEEE Computer Vision and Pattern Recognition (CVPR 2017), Honolulu, 2017. Available at: https://arxiv.org/abs/1609.04802.

Tran L, Yin X, Liu X M. Disentangled representation learning gan for pose-invariant face recognition. In Proceeding of IEEE Computer Vision and Pattern Recognition (CVPR 2017), Honolulu, 2017. Available at: http://cvlab.cse.msu.edu/pdfs/Tran_Yin_Liu_CVPR2017.pdf

Rumelhart D E, Hinton G E, Williams R J. Learning representations by back-propagating errors. Nature, 1986; 323(6088): 533-536.

Lecun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998; 86(11): 2278–2324.

Krizhevsky, Alex, Ilya Sutskever, Geoffrey E. Hinton. Imagenet classification with deep convolutional neural networks. Proceedings of Advances in Neural Information Processing Systems, 2012.

Mikolov T, Karafiát M, Burget L, Černocký J H, Khudanpur S. Recurrent neural network based language model. Eleventh Annual Conference of the International Speech Communication Association,

Chiba, Japan, 2010.

Sundermeyer M, Schlüter R, Ney H. LSTM neural networks for language modeling. Thirteenth Annual Conference of the International Speech Communication Association, Portland, Oregon, USA, 2012.

Abadi M, Barham P, Chen J M, Chen Z F, Davis A, Dean J, et al. TensorFlow: A system for large-scale machine learning. Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI ’16), Savannah, GA, USA, 2016.

Rampasek L, Goldenberg A. Tensorflow: Biology’s gateway to deep learning? Cell Systems, 2016; 2(1): 12–14.

Jia Y Q, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, et al. Caffe: Convolutional architecture for fast feature embedding. Proceedings of the 22nd ACM international conference on Multimedia. 2014. Available at: https://arxiv.org/pdf/1408.5093.pdf.

Yalcin H, Razavi S. Plant classification using convolutional neural networks. Fifth International Conference on Agro-Geoinformatics (Agro-Geoinformatics), IEEE, 2016.

Sladojevic S, Arsenovic M, Anderla A, Culibrk D, Stefanovic D. Deep neural networks based recognition of plant diseases by leaf image classification. Computational Intelligence and Neuroscience, 2016; Article ID 3289801, 11p.

Hanson A, Joel M G, Joy A, Francis J. Plant leaf disease detection using deep learning and convolutional neural network. International Journal of Engineering Science, 2017; 5324.

Tang J L, Wang D, Zhang Z G, He L J, Xin J, Xu Y. Weed identification based on K-means feature learning combined with convolutional neural network. Computers and Electronics in Agriculture, 2017; 135: 63–70.

Srbinovska M, Gavrovski C, Dimcev V, Krkoleva A, Borozan V. Environmental parameters monitoring in precision agriculture using wireless sensor networks, Journal of Cleaner Production, 2015; 88: 297–307.

Ge Z Y, McCool C, Sanderson C, Corke P. Content specific feature learning for fine-grained plant classification. CLEF (Working Notes), 2015. Available at: http://ceur-ws.org/Vol-1391/39-CR.pdf.

Yalcin H. Plant phenology recognition using deep learning: 6th International Conference on Deep-Pheno. Agro-Geoinformatics, IEEE, 2017.

Chen S W, Shivakumar S S, Dcunha S, Das J, Okon E, Qu C, et al. Counting apples and oranges with deep learning: A data-driven approach. IEEE Robotics and Automation Letters, 2017; 2(2): 781–788.

Kussul N, Shelestov A, Lavrenyuk M, Butko I, Skakun S. Deep learning approach for large scale land cover mapping based on remote sensing data fusion. Geoscience and Remote Sensing Symposium (IGARSS), IEEE, 2016. doi: 10.1109/IGARSS.2016.7729043.

Lu H, Fu X, Liu C, Li L G, He Y X, Li N W. Cultivated land information extraction in UAV imagery based on deep convolutional neural network and transfer learning. Journal of Mountain Science, 2017; 14(4): 731–741.

Christiansen P, Hansen M, Steen K, Karstoft H, Jørgensen R. Advanced sensor platform for human detection and protection in autonomous farming. In Precision Agriculture’15; Wageningen Academic Publishers: Wageningen, the Netherlands, 2015; pp.1330–1334.

Steen K A, Christiansen P, Karstoft H, Jørgensen R N. Using deep learning to challenge safety standard for highly autonomous machines in agriculture. Journal of Imaging, 2016; 2(1): 6.

Arkeman Y, Buono A, Hermadi I. Satellite image processing for precision agriculture and agroindustry using convolutional neural network and genetic algorithm. IOP Conference Series: Earth and Environmental Science 54 (2017) 012102. doi: 10.1088/1755-1315/54/1/012102.

Sun Y, Zhu L, Wang G, Zhao F. Multi-input convolutional neural network for flower grading. Journal of Electrical and Computer Engineering, 2017; Article ID 9240407, 8p. https://doi.org/10.1155/ 2017/9240407.

Salman A G, Kanigoro B, Heryadi Y. Weather forecasting using deep learning techniques. Advanced International Conference on Computer Science and Information Systems (ICACSIS), IEEE, 2015.

You J X, Li X C, Low M, Lobell D, Ermon S. Deep Gaussian process for crop yield prediction based on remote sensing data. Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17), pp.4559–4665.

Kasfi K T, Hellicar A, Rahman A. Convolutional neural network for time series cattle behaviour classification. Proceedings of the Workshop on Time Series Analytics and Applications. ACM, 2016.

Rahman A, Smith D, Hills J, Bishop-Hurley G, Henry D, Rawnsley R. A comparison of autoencoder and statistical features for cattle behaviour classification. In IEEE World Congress on Computational Intelligence (IEEE WCCI), 2016; pp.1–7.

Rußwurm M, Körner M. Multi-temporal land cover classification with long shortterm memory neural networks. ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-1/W1, 2017; pp.551–558.

Ienco D, Gaetano R, Dupaquier C, Maurel P. Land cover classification via multi-temporal spatial data by recurrent Neural Networks IEEE Geoscience & Remote Sensing Letters, 2017; 99: 1–5.

Lyu H, Lu H, Mou L. Learning a transferable change rule from a recurrent neural network for land cover change detection. Remote Sensing, 2016; 8(6): 506.

Namin S T, Esmaeilzadeh M, Najafi M, Brown T B, Borevitz J O. Deep phenotyping: Deep learning for temporal phenotype/genotype classification. bioRxiv, 2017. doi: https://doi.org/10.1101/134205.

Minh D H T, Ienco D, Gaetano R, Lalande N, Ndikumana E. Deep recurrent neural networks for winter vegetation quality mapping via multitemporal SAR Sentinel-1. IEEE Geoscience & Remote Sensing Letters, 2017; 99: 1–5.

Chai L N, Qu Y H, Zhang L X, Liang S L, Wang J D. Estimating time-series leaf area index based on recurrent nonlinear autoregressive neural networks with exogenous inputs International Journal of Remote Sensing, 2012; 33 (18): 5712–5731.

Chen B Q, Wu Z X, Wang J K, Dong J W, Guan L M, Chen J M, et al. Spatio-temporal prediction of leaf area index of rubber plantation using HJ-1A/1B CCD images and recurrent neural network. ISPRS Journal of Photogrammetry and Remote Sensing, 2015; 102: 148–160.

Biswas S K, Sinha N, Purkayastha B, Marbaniang L. Weather prediction by recurrent neural network dynamics. International Journal of Intelligent Engineering Informatics, 2014; 2(2-3): 166–180.

Zaytar M A, El Amrani C. Sequence to sequence weather forecasting with long short-term memory recurrent neural networks. International Journal of Computer Applications, 2016; 143(11): 7–11.

Lu Z, Chai L N, Liu S M, Cui H Z, Zhang Y H, Jiang L M, et al. Estimating time series soil moisture by applying recurrent nonlinear autoregressive neural networks to passive microwave data over the Heihe River Basin, China. Remote Sensing, 2017; 9(6): 574.

Tzeng Y C, Fan K T, Lin C Y, Lee Y J, Chen K S. Estimation of soil moisture dynamics using a recurrent dynamic learning neural network. Geoscience and Remote Sensing Symposium, 2012; 88(8): 1251–1253.

Palangpour P, Venayagamoorthy G K, Duffy K. Recurrent neural network based predictions of elephant migration in a South African game reserve. International Joint Conference on Neural Networks, 2006: pp.4084–4088.

Demmers T G M, Gauss S, Wathes C M, Cao Y, Parsons D J. Simultaneous monitoring and control of pig growth and ammonia emissions. IX International Livestock Environment Symposium, 2012; C-1323.

Wahlberg F, Wilkinson T, Brun A. Historical manuscript production date estimation using deep convolutional neural networks. 15th International Conference on Frontiers in Handwriting Recognition (ICFHR), IEEE,

Ledig C, Theis L, Huszar F, Caballero J, Cunningham A, Acosta A, et al. Photo-realistic single image super-resolution using a generative adversarial network. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, Hawaii, USA, 2017; pp.105–114.

Barth R, IJsselmuiden J M M, Hemming J, van Henten E J. Optimising realism of synthetic agricultural images using cycle generative adversarial networks. Proceedings of the IEEE IROS workshop on Agricultural Robotics/Kounalakis, Tsampikos, van Evert, Frits, Ball, David Michael, Kootstra, Gert, Nalpantidis, Lazaros, Wageningen: Wageningen University & Research, 2017; pp.18–22. http://library.wur.nl/WebQuery/wurpubs/ 533105

Hu K, Qi K, Guan Q, Wu C, Yu J, Qing Y, et al. A scientometric visualization analysis for night-time light remote sensing research from 1991 to 2016. Remote Sensing, 2017; 9(8): 802. doi: 10.3390/ rs9080802.

Garfield E. From the science of science to scientometrics visualizing the history of science with HistCite software. Journal of Informetrics, 2009; 3(3): 173–179.

McCown R L. Changing systems for supporting farmers' decisions: problems, paradigms, and prospects. Agricultural Systems, 2002; 74(1): 179–220. doi: https://doi.org/10.1016/S0308-521X(02)00026-4.

Grinblat G L, Uzal L C, Larese M G, Granitto P M. Deep learning for plant identification using vein morphological patterns. Computers and Electronics in Agriculture, 2016; 127: 418–424. doi: 10.1016/j.compag.2016.07.003.

Ding W, Taylor G. Automatic moth detection from trap images for pest management. Computers and Electronics in Agriculture, 2016; 123: 17–28. doi: https://doi.org/10.1016/j.compag.2016.02.003.

Chen C. CiteSpace II: Detecting and visualizing emerging trends and transient patterns in scientific literature. Journal of the American Society for Information Science and Technology, 2006; 57(3): 359–77.

Blei D M, Ng A Y, Jordan M I. Latent dirichlet allocation. Journal of Machine Learning Research, 2003; 3(1): 993–1022.

Chen C, Leydesdorff L. Patterns of connections and movements in dual-map overlays: A new method of publication portfolio analysis. Journal of the Association for Information Science and Technology, 2013; 65(2): 334–51. doi: 10.1002/asi.22968.

Blondel V D, Guillaume J-L, Lambiotte R, Lefebvre E. Fast unfolding of communities in large networks. Journal of Statistical Mechanics: Theory And Experiment, 2008; 2008(10): P10008. doi: 10.1088/1742-5468/ 2008/10/P10008.

Gwan J, Moon H H, Tae J, Syed K, Hassan I, Dang M, et al. Deep convolutional neural network for classifying Fusarium wilt of radish from unmanned aerial vehicles. Journal of Applied Remote Sensing, 2017; 11(4). doi: 10.1117/1.JRS.11.042621.

Dyrmann M, Midtiby H S, Mortensen A K. Pixel-wise classification of weeds and crops in images by using a fully convolutional neural network. Proceedings of the International Conference on Agricultural Engineering, Aarhus, Denmark, 2016.

Grinblat G L, Uzal L C, Larese M G, Granitto P M. Deep learning for plant identification using vein morphologicalpatterns. Computers and Electronics in Agriculture, 2016; 127: 418–424.




Copyright (c) 2018 International Journal of Agricultural and Biological Engineering



2023-2026 Copyright IJABE Editing and Publishing Office