Research and development in agricultural robotics: A perspective of digital farming

: Digital farming is the practice of modern technologies such as sensors, robotics, and data analysis for shifting from tedious operations to continuously automated processes. This paper reviews some of the latest achievements in agricultural robotics, specifically those that are used for autonomous weed control, field scouting, and harvesting. Object identification, task planning algorithms, digitalization and optimization of sensors are highlighted as some of the facing challenges in the context of digital farming. The concepts of multi-robots, human-robot collaboration, and environment reconstruction from aerial images and ground-based sensors for the creation of virtual farms were highlighted as some of the gateways of digital farming. It was shown that one of the trends and research focuses in agricultural field robotics is towards building a swarm of small scale robots and drones that collaborate together to optimize farming inputs and reveal denied or concealed information. For the case of robotic harvesting, an autonomous framework with several simple axis manipulators can be faster and more efficient than the currently adapted professional expensive manipulators. While robots are becoming the inseparable parts of the modern farms, our conclusion is that it is not realistic to expect an entirely automated farming system in the future.


Introduction
Modern farms are expected to produce more yields with higher quality at lower expenses in a sustainable way that is less dependent on the labor force.Implementation of digital farming and site-specific precision management are some of the possible responses to this expectation, which depends not only on the sensor technology but the continuous collection of field data that is only feasible through proper utilization of agricultural robots.Agricultural scientists, farmers, and growers are also facing the challenge of producing more food from less land in a sustainable way to meet the demands of the predicted 9.8 billion populations in 2050 [1] .That is equivalent of feeding a newly added city of 200 000 people every day.Integration of digital tools, sensors, and control technologies has accelerated design and developments of agricultural robotics, demonstrating significant potentials and benefits in modern farming.
These evolutions range from digitizing plants and fields by collecting accurate and detailed temporal and spatial information in a timely manner, to accomplishing complicated nonlinear control tasks for robot navigation.Autonomous guided tractors and farm machinery equipped with local and global sensors for operating in row-crops and orchards have already become mature.Examples include the John Deere iTEC Pro (Deere & Company, Moline, Illinois) which uses Global Navigation Satellite System for steering control, and the Claas autonomous navigation (Harsewinkel, Ostwestfalen-Lippe, Germany) which offers Cam Pilot steering and 3D computer vision in addition to the GPS-based control to follow features on the ground.Agricultural field robots and manipulators have become an important part in different aspects of digital farming [2] and precision agriculture [3] .With the advances in controls theory, applications of these robots in digital farming have shown growing interest towards automation, changing the traditional field activists to high-tech industrial tasks that are attracting investors, professional engineers, and companies.While many are still in the prototype phase, these robots are now capable of performing various farming operations, including crop scouting [4] , pest and weed control [5] , harvesting [6][7][8][9][10] , targeted spraying [11,12] , pruning [13,14] , milking [15,16] , Phenotyping [17,18] , and sorting [19] .Unlike the industrial case, these applications can be extremely challenging to be fully automated.An agricultural robot is subjected to an extremely dynamic environment, and yet expected to touch, sense, or manipulate the crop and the surroundings in a precise manner which makes it necessary to have the minimal amount of impact while increasing efficiency [20] .Although industrial robotic platform with precision accuracy and speed are available, their application in agriculture is limited due to what we refer to as unstructured environments and uncertain tasks which impose great challenges.For example, the demand for off-season cultivation of fruits and vegetables require different aspects of automation and robotics in closed-field plant production environments like greenhouses [21] .A field robot with spraying, de-leafing, and harvesting manipulator and end-effector for such tasks in a dynamic, complex, and uncertain environment should take into account the different arrangements of plant sizes and shapes, stems, branches, leaves, fruit color, texture, obstacles, and weather influences in order to operate efficiently in the real world condition.In the case of harvesting for example, the sensing mechanism has to identify the ripeness of fruits in the presence of various disturbances in an unpredicted heterogeneous environment, while the actuation mechanism should perform motion and path planning to navigate inside the plant system or tree canopy with minimum collisions for grasping and removing the soft fruit delicately.This is by far more challenging compared to an industrial robot in charge of picking and placing a solid bolt in an assembly line.
The organization of this paper is as follow: In Section 2 we have provided a literature review on the research and development in agricultural robotics followed by separated discussions focused on weed control, field scouting, and harvesting robots.Section 3 highlights the perspective of agricultural robotics and the opportunities for digital farming and virtual orchards.Section 4 extends our discussion on the challenges of digitalization, automation, and optimization of robotics for precision agriculture.A summary of findings and conclusions are presented in section 5.
In addition to these, several virtual experimentation frameworks have been developed for agricultural robots.An example includes the work of [76] in which a generic high-level functionality was provided for easier and faster development of agricultural robots.
In another attempt, a customized software platform called ForboMind [77] was introduced to support field robots for precision agriculture task with the objective to promote reusability of robotic components.ForboMind is open-source, and support projects of varying size and complexity, facilitate collaboration for modularity, extensibility, and scalability.In order to experiment with vision sensors and agricultural robots [7] , created a completely simulated environment in V-REP (Coppelia Robotics) [78] , ROS [79] , and MATLAB (Mathworks, Natick, MA, USA) for improvement of plant/fruit scanning and visual servoing task through an easy testing and debugging of control algorithms with zero damage risk to the real robot and to the actual equipment.Example solutions addressing robotic harvesting included eye-in-hand look-and-move configuration for visual servo control [49,[80][81][82] , optimal manipulator design and control [29,38] , end-effector and gripper design [8,83] , stability tests for robot performance analysis in the dense obstacle environments [84] , motion planning algorithms [85] , and orchard architecture design for optimal harvesting robot [6] .Improvements in vision-based control system [7,48,49,86,87] have enabled several applications of robotic manipulators for greenhouse and orchard tasks and have contributed to the decrease in workload and labor's fatigue while improving the efficiency and safety of the operations.These achievements were considered a challenge in the earlier agricultural robotics works [88][89][90] .
Agricultural field robots [91] on the other hand contribute to increasing the reliability of operations, improved soil health, and improved yield.They are generally equipped with two or multiple sensors and cameras for navigation control, simultaneous localization and mapping, and path planning algorithms [92][93][94] .Some of the earlier attempts for developing agricultural field robots prototypes can be found in the works of [95-98].The automated harvesting platform shown in Figure 1 is one of the most recent achievements in the field of agricultural robotics.
It was introduced by the SWEEPER EU H2020 project consortium -(www.sweeper-robot.eu)on July 4, 2018.It is an assembly of an autonomous mobile platform with Fanuc LRMate 200iD robot manipulator (Fanuc America Corporation, Rochester Hills, MI) holding an end-effector and catching device for fruit harvesting.The ultimate goal of the Sweeper project is to put the first working sweet pepper harvesting robot on the market.Using the camera system mounted on the end-effector, the SWEEPER scans plants looking slightly upwards for detecting mature fruits (the robot observe the bottom part of the peppers to determine the fruit maturity).
The camera and sensors setup is completely independent of the surrounding light conditions and provide information about color images and distance maps that are used for fruit detection, localization, and maturity classification.The SWEEPER robot has been trained to detect obstacles such as leaves and plant stems in the images.The training process was accelerated using simulated artificial pepper plant models and deep learning network algorithms.Once the robot detects a pepper fruit, information about its location is used to perform path planning for the robotic arm trajectory.Because of the limited moving space between the planting rows calculation of this trajectory can be very complex.The robot then employs visual servo control to reach the peduncle of the spotted peppers.The robot camera takes images from different angles so that the arm approaches the pepper in such a direction that the stem is always on the back side of the pepper.A small cutting tool is positioned just above the pepper which cuts the peduncle while the cutting tool is moving downward.This separates the pepper from the plant's stem and drops it into a catching device which is moved toward the pepper bin by the robotic arm.It is notable that the SWEEPER can only harvest fruits that are located on the front side of the plants and stems.A conveyer belt is said to be added to the robot in order to convey harvested peppers to a standard pepper trolley.Multiple robots and trolley will be parts of a fully automated post-harvest logistic management system.The robot will exploit its harvesting skill in full in a single stem row cropping system.
The most suitable yellow existing variety was used during the SWEEPER test.According to the project website, for single row growing system, the performance of SWEEPER evaluated with only fruits that were on the front side of stems was respectively 62% and 31% in the modified and commercial crop.In general, SWEEPER has a success rate of 49% in harvesting ripe fruits with modified crop, and only 20% with the commercial (current greenhouse growing) system.The average time to harvest one fruit with SWEEPER is between 18 and 25 seconds compromising 4.73 s for platform movement, 3.71 s for fruit localization, 3.02 s for obstacle localization, 4.03 s for visual servoing, 2.22 s for fruit detaching, and 7.77 s for dropping fruit in container (data extracted from SWEEPER website).The SWEEPER project team has announced in their website that they have also achieved a harvest time of less than 15 seconds (excluding platform movement) in laboratory experiments.It is expected that results of projects like this will serve as input for the development of a new fully optimized and automated fruit production system for the greenhouse horticulture sector.
Source: Sweeper EU H2020 project consortium -www.sweeper-robot.eu.For the purpose of this paper we provide a general review of the recent advances in agricultural robotics, with focus on those that employ high-tech sensors, artificial intelligence, machine learning, and simulation environments for (i) weed control and targeted spraying, (ii) field scouting and data collection, and (iii) automated harvesting.We then extend our discussion to introduce some of the most widely used simulation software and virtual platforms that can be adapted to accelerate the design of agricultural robots, improve operational performances, and evaluate control capabilities of the actual hardware.

Weed control and targeted spraying robots
One of the main aspects of agricultural robotics is concerned with the substitution of the human workforce by field robots or mechanized systems that can handle the tasks more accurately and uniformly at a lower cost and higher efficiency [6,[99][100][101][102][103] .Weed control and precise spraying are perhaps the most demanded applications for agricultural field robots.In this regard, targeted spraying [104] with robots for weed control application has shown acceptable results and reduced herbicide use to as little as 5%-10% compared to blanket spraying [105] .
While still not fully commercialized, various promising technologies for weed robots have been introduced and implemented over the past 10 years as the results of interdisciplinary collaborative projects between different international research groups and companies.Some of the well-known names that are actively involved in the research and development for various types of weed control robots are the Wageningen University and Research Center (The Netherlands), Queensland University of Technology, the University of Sydney, Blue River Technologies (Sunnyvale, CA, USA), Switzerland's ecoRobotix (Yverdon-les-Bains, Switzerland), and France's Naio Technologies (Escalquens, France).For example a flexible multipurpose farming and weeding robot platform named BoniRob [18,106] (shown in Figure 2a) was developed as a joint project between the University of Osnabrueck, the DeepField Robotics start-up, Robert Bosch company, and the machine manufacturer Amazonen-Werker.The available time, labor, equipment, costs, and types of weeds and the areas infested need to be considered when planning a weed control program.For such a robot to be efficient, it should be able to not only substitute the tedious manual weed removal task, but also decreases the use of spraying agrochemical and pesticide on the field.Figure 2 shows: (a) BoniRob [18,106] : an integrated multipurpose farming robotic platform for row crops weed control developed by interdisciplinary teams which is also capable of creating details map of the field, (b) AgBot II [107] : an innovate field robot prototype developed by the Queensland University of Technology for autonomous fertilizer application, weed detection and classification, and mechanical or chemical weed control, (c) Autonome Roboter [108] : a research effort robot developed by Osnabrück University of Applied Sciences for weed control, (d) Tertill [109] : a fully autonomous solar powered compact robot developed by FranklinRobotics for weed cutting, (e) Hortibot [110] : a robot developed by the Faculty of Agricultural Sciences at the University of Aarhus for transporting and attaching a variety of weed detection and control tools such as cameras, herbicide and spraying booms, (f) Kongskilde Robotti [111] : a robotic platform equipped with drive belt operating based on the FroboMind software [77] that can be connected to different modules and implements for automated and semi-automated mechanical weed control, precision seeding, furrow opening and cleanings, (g) RIPPA [112] : a solar-powered Robot for Intelligent Perception and Precision Application developed by the Australian Centre for Field Robotics at Sydney University, and (h) spray robot developed by HollandGreenmachine for smart chemical application in greenhouses.Some of these robots can reduce weed chemical use by 80%-90% [18,107,112] .a. BoniRob [18,106] b.AgBot II [107] c.Autonome Roboter [108] d.Tertill [109] Source: Deepfield Robotics Source: Queensland University of Technology Source: Osnabrück University Source: franklinrobotics.come. Hortibot [110] f.Kongskilde Robotti [111] g.RIPPA [112] h.Spray robot Image credit: technologyreview.comImage credit: conpleks.comSource: The University of Sydney Source: Hollandgreenmachine In order to apply chemical directly to the weed's vascular tissue, a direct chemical application end effector is required to cut the weed's stem and spread the chemical on the cut surface.An example of such an application can be found in [113] where a proto-type weed control robot was developed to spray weeds in cotton plants in the seed line.A real-time intelligent weed control system was introduced in [114] for selective herbicide application to in-row weeds using machine vision and chemical application.A mini-robot to perform spraying activities based on machine vision and fuzzy logic has been described in [115,116].More examples of autonomous vehicle robot for spraying the weeds can be found in [117-119] and [90] and [114].Development of an autonomous weeding machine requires a vision system capable of detecting and locating the position of the crop.Such a vision system should be able to recognize the accurate position of the plant stem and protects it during the weed control [120] .A near-ground image capturing and processing technique to detect broad-leaved weeds in cereal crops under actual field conditions has been reported in the work of [121].Here the researchers proposed a method that uses color information to discriminate between vegetation and background, whilst shape analysis techniques were applied to distinguish between crop and weeds.Shape features of the radish plant and weed were investigated by [122].They proposed a machine vision system using a charge coupled device camera for the weed detection in a radish farm resulting 92% success rate of recognition for radish and 98% for weeds.A combined method of color and shape features for sugar beet weed segmentation was proposed by [98] with a 90% success rate in classification.This rate increased to 96% by adding two shape features.Another approach extracted a correlation between the three main color components R, G and B which constitute weeds and sugar beet color classes by means of discriminant analysis [123] .Their method resulted in different classification success rates between 77 and 98%.The segmentation of weeds and soybean seedlings by CCD images in the field was studied by [124].Texture features of weed species have been applied for distinguishing weed species with grass and broadleaf classification accuracies of 93% and 85%, respectively [125] .Textural image analysis was used to detect weeds in the grass [126] .Gabor wavelet features of NIR images of apples were extracted for quality inspection and used as input to kernel PCA [127] .Kernel PCA first maps the nonlinear features to linear space and then PCA is applied to separate the image Gabor wavelet (5 scales and 8 orientations) combined with kernel PCA had the highest recognition rate (90.5%).Spray robots for weed control have been developed with vertical spray booms that increase the deposition in the canopy [128][129][130] .Some of the emerging technologies are the self-propelled vehicles such as Fumimatic® (IDM S.L, Almería, Spain) and Tizona (Carretillas Amate S.L., Almería, Spain), or autonomous field robots such as Fitorobot (Universidad de Almería, Cadia S.L., Almería, Spain) that have been designed specifically to navigate inside fields that has loose soil and operate in situations where a large number of obstacles are present [130] .Some of these robots are based on inductive sensors for following metal pipes that are buried in the soil.Studies that reports autonomous robot navigation inside greenhouse environments are slim [116,118,131,132] .A fixed-position weed robot was presented by [133] which is interfaced to a standard belt-conveyor displacement system and provides the robot with pallets containing the crops.Artificial neural networks have also been used by many researchers to discriminate weeds with machine vision [134,135] .For example, BoniRob [18,106] uses AI to differentiate between weeds and plants and then mechanically destroys the detected weeds using a custom-built mechanism called "ramming death rod".Different control modules of BoniRob are connected by Ethernet and communicate using TCP/IP.This platform has 16 degrees of freedom (DOF) realized by different electro-motors and hydraulic cylinder actuation.Each of the 4 wheels is driven by separate motors and can be steered independently (motor controllers are connected by a CAN bus).These reviews indicate that a fully commercial robotic platform for the elimination of weeds has not been realized yet.In addition, most of the research works in the area of robotic weed control are applicable prior to the plant growth or in some cases when the main plant height is between 0.2-0.3 m.

Field scouting and data collection robots
Field scouting robots face various interdisciplinary challenges for providing reliable data and measurements that can be used and processed by precision agriculture and crop models.Other than the challenges of inherent physical and biological variability involved with farm fields and orchards, scouting robot platforms are expected to be flexible, multipurpose and affordable to be considered viable for use in commercial scale.If successfully integrated and implemented, these robots can play a key role in reducing production cost, increasing productivity, quality, and enabling customized plant and crop treatments.Development of scouting robots for the purpose of data collection and modern farming incorporates extensive use of advanced sensors for precision agriculture [136,137] in order to generate valuable results while performing automatic and accurate navigation control, manipulator control, obstacle avoidance, and three-dimensional environment reconstructions.For example, an autonomous field survey mobile robot platform with custom manipulator and gripper was proposed [138] to carry imaging sensors and GPS devises for autonomous navigation and data collection inside greenhouses and open-field cultivation environments (Figure 3a-3c).Various multi-spectral imaging devices and LiDAR sensors are reported to have been installed and used with modified mobile robot platforms for automated monitoring and building reconstructed 3D point clouds for generating computer images of trees and plants [69,[139][140][141] such as those shown in Figure 3d.

Actual Simulated Actual Simulated
a. A prototype surveillance field robot [138], A daptiveAgroTech.com b.OSCAR field survey robot, Inspectorbots.com c. Husky UGV for field scouting and 3D mapping, Clearpathrobotics.com d.Point cloud and detected maize plants [69] , and 3D point clouds of vineyard created by VinBotRobotnik Automation (www.robotnik.eu) [140]gure 3 Examples of a prototype and professional field robots for scanning and 3D reconstruction of plants and environment Some of the most advanced robotic technology for automated field scouting and data collection are shown in Figure 4 including (a) Trimbot2020 [142] , an outdoor robot based on a commercial Bosch Indigo lawn mower platform and Kinova robotic arm for automatic bush trimming and rose pruning, (b) Wall-Ye [143] , a prototype vineyard robot for mapping, pruning, and possibly harvesting the grapes (wall-ye.com),(c) Ladybird [144,145] , an autonomous multipurpose farm robot for surveillance, mapping, classification and detection for different vegetables, (d) MARS [146,147] : the mobile agricultural robot swarms are small and stream-lined mobile robot units that have minimum soil compaction and energy consumption and aim at optimizing plant specific precision agriculture, (e) SMP S4: a surveillance robot for bird and pest control developed by SMP Robotics (smprobotics.com),(f) Vine agent, a robot equipped with advanced sensors and artificial intelligence to monitor the field for plant's health assessment developed at the Universitat Politècnica de València, (g) HV-100 Nursery Bot, a light weight robot developed by Harvest Automation for moving of plants and potted trees in greenhouses and small orchards developed by Harvest Automation (harvestai.com/),(h) VinBot [59,148] : an all-terrain mobile robot with advanced sensors for autonomous image acquisition and 3D data collection from vineyar for yield estimation and information sharing, (i) Mantis, a flexible general purpose robotic data collection platform equipped with RADAR, liDAR, panospheric, stereovision, and thermal cameras [32] , and (j) GRAPE, a Ground Robot for vineyard monitoring and ProtEction funded by the European Union's for smart autonomous navigation, plant detection and health monitoring, and manipulation of small objects [149] .a. TrimBot [142] b.Wall-Ye vinyard robot c.Lady bird [144] j. MARS[146,147] e. SMP S4 trimbot2020.orgwall-ye.comUniv. of Sydney echord.eu/mars smprootics.comf. VineRobot g. HV-100HV-100 Nursery Bot h.VinBot [59,148] i. Mantis [32] d.GRAPE vinerobot.euharvestai.comvinbot.eu/ Univ. ofSydney grape-project.eu Examples of general purpose robots for field scouting and data collection

Harvesting robots
Traditional harvesting of fruits and vegetables for fresh market is a labor-intensive task that demands shifting from tedious manual operation to a continuously automated harvesting.Increasing the efficiency and reducing labor dependents of harvesting will ensure high-tech food production yield and competitiveness.In spite of the advances in agricultural robotics, million tons of fruits and vegetables are still hand-picked every year in open-fields and greenhouses.Other than the high labor cost, the availability of the skilled workforce that accepts repetitive tasks in the harsh field conditions impose uncertainties and timeliness costs.For robotic harvesting to be cost-effective, fruit yield needs to be maximized to compensate the additional automation costs.This leads to growing the plants at higher densities which make it even harder for an autonomous robot to simultaneously detect the fruit, localize and harvest it.In the case of sweet pepper fruit, with an estimated yield of 1.9 million tons/year in Europe, reports indicate that while an average time of 6 seconds per fruit is required for automated harvesting, the available technology has only achieved a success rate of 33% with an average picking time of 94 seconds per fruit [25] .For cucumber harvesting, a cycle time of 10 seconds was proven to be economically feasible [38] .Only in Washington State, 15-18 billion apple fruits are harvested manually every year.An estimated 3 million tons of apples is reported to have been produced in Poland in 2015 [150] , out of which one-third are delicate fruits and are less resistant to bruising from mass harvester machines.Also in Florida, where the current marketable yield of sweet pepper fruits in open-field cultivation is 1.6 to 3.0 with a potential yield of 4 lb/ft 2 in passive ventilated greenhouses [151] , manual harvesting is still the only solution.
Therefore, development of an automated robotic harvesting should be considered as an alternative method to address the associated labor shortage costs and timeliness.A fully automated robotic harvester will contribute to solving some of the today's major grower issues, such as labor costs, labor availability, food safety and quality.It also plays an essential role in improving the interactions between human, machine, and plants [131] .For example, the prevention of musculoskeletal disorders in manual harvesting operations in Dutch greenhouses has motivated various researchers for replacement of human labor by automatons robot for picking cucumber [33] and sweet pepper [7] fruits.A functional model was then introduced [23] in the field test of an autonomous robot for de-leafing cucumber plants grown in a high-wire cultivation system.Field results showed that the de-leafing robot spent an average time of 140 seconds for two leaves plants, which was 35 times longer than manual leaf picking per plant [23] .
Research and development in robotic harvesting date back to the 1980s, with Japan, The Netherlands, and the USA as the pioneer countries.The first studies used simple monochrome cameras for fruit detection inside the canopy [152] .Other than the visible light RGB cameras [41,153] and the ultrasonic radar sensors that are commonly used for object detection due to their affordable cost [154] , advances in the sensing and imaging technology have led to the employment of sophisticated devices such as infrared [47] , thermal [155] , hyperspectral cameras [156] , LiDAR [32,39,40,157] , or combination of multi-sensors [158] that are adopted with novel vision-based techniques for extracting spatial information from the images for fruit detection, recognition, localization, and tracking.A common approach in fruit detection and counting [153] is by using a single viewpoint, as in the case of a cucumber harvesting robot [23] , or multiple viewpoints [32] with additional sensing from one or multiple vision sensors that are not located on the robot [159] .Examples of the recent achievements include automatic fruit recognition from multiple images [160] or based on the fusion of color and 3D feature [161] , multi-template matching algorithm [162] , symmetry analysis [163] , combined color distance method and RGB-D data analysis for apples [164] and sweet-peppers [41] , stereo vision for apple detection [165,166] , and the use of convolutional neural networks [167] and deep learning algorithms for fruit detection and obstacle avoidance in extremely dense foliage [54,168] .Some of the challenges to be addressed in designing of a complete robotic harvesting are the simultaneous localization of fruit and environment mapping, path planning algorithms, and the number of detectable and harvestable fruits in different plant density conditions.Significant contributions have been made by various research groups to address these challenges, however there is currently no report of a commercial robotic harvesting for fresh fruit market [169] , mainly due to the extremely variable heterogeneous working condition and the complex and unpredicted tasks involved with different fruit and plant scenario.The function of a harvesting robot can be separated into three main sections as sensing (i.e., fruit recognition), planning (i.e., hand-and-eye coordination) and acting (i.e., end-effector mechanism for fruit grasping) [170] .
Theoretical and applied research on robotic harvesting of fruits and vegetable are huge.Figure 5 shows some of the efforts that resulted in building actual robotic harvesting platforms, including (a) Harvey [28] : an autonomous mobile robot platform with UR5 manipulator for harvesting sweet peppers grown in greenhouses and other protect cultivation systems, (b) the CROPS harvesting platform for sweet pepper [27,171] , (c) the SWEEPER platform (developed by the Sweeper EU H2020 project consortium, www.sweeper-robot.eu)with a Fanuc LRMate 200iD robot manipulator (Fanuc America Corporation, Rochester Hills, MI) and a custom-built gripper and catching mechanism for sweet pepper harvesting, (d) the Energid robotic citrus picking system (Bedford, MA), (e) the citrus harvesting robot [48,49,155] developed at the University of Florida which uses a custom built gripper mounted on the Robotics Research manipulator model 1207 (Cincinnati, Ohio), (f) the DogTooth strawberry robot (Great Shelford, Cambridge, UK), (g) the Shibuya Seiki robot that can harvest strawberry fruits every 8 seconds, (h) a tomato harvesting robot from Suzhou Botian Automation Technology Co., Ltd (Jiangsu, Suzhou, China), (i) a cucumber harvesting robot developed at the Wageningen University and Research Center [35,38] , (j) an apple harvesting robot [172] with custom built manipulator mounted on top of a modified crawler mobile robot, (k) one of the first manipulators developed for the CROPS project [171] and modified for apple harvesting, (l) a linear actuator robotic system for apple picking developed by ffrobotics (Gesher HaEts 12, Israel), (m) a vacuum mechanism robot for apple picking from AbundantRobotics (Hayward, CA, USA), (n) the UR5 manipulator with a soft robotic universal gripper for apple harvesting developed at the University of Sydney, and, (n) an apple catching prototype robot [173][174][175] developed at the Wachington State University.Most of these projects have used eye-in-hand look-and-move configuration in their visual servo control.Other than the issues with frame transformation, this solution is not promising if the fruit is heavily occluded by the high-density plant leaves [176] .Obviously, the final robot prototype needs to be relatively faster for mass-harvest, with an affordable cost for greenhouse growers.Swarms of simple robots with multiple low-cost cameras and innovative soft robotic grippers [177] , or human-robot collaboration are the research topics to solve the facing challenges in robotic harvesting that current technology cannot overcome.These approaches can significantly improve the processing time of multiple fruit detection in the high-density plants, and provide ground truth results over time for machine learning algorithms based on human-operators experience.In fact, a promising solution to efficient robotic harvesting is not through a single robot manipulator.Results of simulation studies have revealed that single arm robots for rapid harvesting are still far beyond realization, and failed mainly due to the "sensing and moving" action in high vegetation density.In this approach, even if the fruit localization is accurate, and the robot control calculates an optimum trajectory to reach the fruit without receiving additional sensing feedback from the camera, the moment it enters into the dense plant canopy it disrupts the exact location of the target fruit.a. Harvey [28] b.CROPS [27,171] c.SWEEPER d.Energid citrus picking system e.Citrus robot [48,49,155] Queensland Univ. of Technology crops-robots.eusweeper-robot.euEnergid technologies University of Florida f.DogTooth g.Shibuya Seiki h.Tomato harvesting robot i. cucumber robot [35,38] j.Apple harvesting robot [172] www.dogtooth.techshibuya-sss.co.jp szbotian.com.cnWageningen UR k.Apple harvesting [178] l.Apple picker m.Apple picking vacuum n.UR5 apple robot o.Apple catching [173][174][175]

Agricultural robotics and digital farming
Agricultural robotics is a promising solution for digital farming and for handling the problems of workforce shortage and declining profitability.Initial tests with one of the most recent technologies available for automated harvesting (the Harvey [28] robot) has already shown a success rate of 65% and detachment rate of 90% for sweet pepper harvesting in real planting scenario where no leaves and occluded fruits were trimmed or removed.Field agent robots that autonomously monitor and collect data empower growers with real-time detailed information about their crops and farms, revealing upstream images for making data-driven decisions.Agricultural robotic is taking farming practices to a new phase by becoming smarter, detecting sources of variability in the field, consuming less energy, and adapting their performances for more flexible tasks.They have become an integral part of the big picture in the future production of vegetable and crops, i.e., growing plants in space or development of robotized plant factories for producing vegetables in Antarctica.The trend in food production is towards automated farming techniques, compact Agri-cubes, and cultivation systems that have the minimum human interface where skilled workforce are being replaced with robotic arms and mobile platforms.In this context digital farming have integrated new concepts and advanced technologies into a single framework for providing farmers and stakeholders with a fast and reliable method of real-time observations at the plant level (i.e., field data collection and crop monitoring) and acting at a more precise scale (i.e., diagnostics, strategic decision-making, and implementing).Digital farming is about collecting high-resolution field and weather data using ground-based or aerial based sensors, transmitting these data into a central advisory unit, interpreting and extracting information, and providing decisions and actions to the farmers, field robots, or agro-industries.Examples include thermal-RGB imaging system [179] for monitoring of plant and soil for health assessment, creation of information maps (i.e., yield and density maps), and data sharing.Implementation of digital farming practices result in a sustainable, efficient, and stable production with a significant increase in yield.Some of the technologies involved in digital farming include the Internet of Thing [180] , big data analysis [181] , smart sensors [182] , GPS and GIS, ICT [183] , wireless sensor networks [184,185] , UAV [186][187][188] , cloud computing [189][190][191] , simulation software [192][193][194][195] , mapping applications [196,197] , virtual farms [198][199][200] , mobile devices [201][202][203][204] , and robotics.A conceptual illustrating of digital farming and its relationship with agricultural robotics is provided in Figure 6, showing that the collected data by the robot agents are sent to a cloud advisory center for decision makings.The actions are then implemented quickly and accurately by the use of robots or other automated machinery, sending operational updates and notification feedbacks to the farmers and agro-industry sections.
This system of computer-to-robot communication combined with the sophisticated simulation software, analytics applications, and data sharing platforms offers a much smoother control over farming operations.In addition, it provides farmers with details of historical field data for improving their performances and optimizing crop yields for specific plots, or even developing new business models.Source: www.AdaptiveAgroTech.com Figure 6 A conceptual illustration of digital farming and virtual orchards with emphasize on the role of agricultural robotics A key consideration before utilization of field robots in large-scale farms is to take into account the cost-benefit analysis for selecting the number and size of the robots that are required for a specific operation.Digital farming can address this by having simulated robots work in the 3D reconstructed field, orchards, or plantations that have been created using UAV imagery and photogrammetry software.With this approach, even people with limited experience or knowledge about crop production, data collection, and analytical methods could connect their farms to a network, share their field information, and receive prescriptions.Using digital farming, growers can collect valuable information about their fields that were previously ignored or used to be measured by the use of traditional scouting methods.For example, detailed measurements about height and size of each tree in a citrus orchard, nutrient stress, the required time for robot navigation inside the orchard, estimated time for robotic harvesting of a tree, and much more can be extracted from a virtual orchard.While this integration might still seem too ambitious to be widespread in many regions, it can serve as a prophetic awareness for a perceptive outlook, offering new insights that enhance the ability for a modern farming system.
Developments of a new generation of agricultural robots that can easily and safely cooperate to accomplish agricultural tasks become necessary.Heavy tractors and machinery used today compacts the soil, which over time severely deteriorates the fertility of the soil.This is a significant threat to soil in Europe.Compacted soils require more than a decade of expensive treatment to recover its fertility.The problem can be solved by replacing heavy tractors with a number of smaller vehicles that can treat crop fields just as well and without compacting of the soil.However, that scenario requires a human supervisor/operator for each vehicle that is very expensive.A technology is then required to enable a single farmer to supervise and operate a team of these automated vehicles.This includes the development of a mission control center and intelligent coverage path planning algorithms to enable team members to communicate and cooperate, and solve a range of agricultural tasks in a safe and efficient way.One of the topics that have been proposed by many researchers for a long time is the exotic concept of Multiple Robots that can work together to accomplish a specific farming task.The idea is to employ artificial intelligence and genetic algorithms where multiple robots are programmed to collaborate with each other and form an ecosystem.This approach becomes even more useful when robots begin learning from each other and improve their performance over time.For example, a swarm of robots can contribute to the creation of nutrient maps by collecting soil samples and communicating with a cloud advisory center for executing proper action on the go.The efficiency of this process may not be great in the beginning, but the performances can be improved over time by having deep learning algorithms that emphasize the so-called good-behavior and punish the bad-behavior of each robot.
These robots have great advantageous for digital farming.For example, a heterogeneous Multi-Robot system compromising a ground mobile platform and an aerial vehicle (Figure 7) for mapping environmental variables of greenhouses has been simulated in the Unity3D 5.2.1 game engine [205] .This robot can measure the temperature, humidity, luminosity and carbon dioxide concentration in the ground and at different heights.Some of the relevant challenges in developing multi-robot sensory systems are mission planning and task allocation, obstacle avoidance, the guidance, navigation and control of robots in different farming scenarios.a. Air-ground Multi-Robot platform and its simulation in Unity3D [205] b.Project MARS [146,147] c.A Swarm of Nano Quadrotors Source: TheDmel Youtube channel After 20 years of research in precision agriculture there are nowadays many types of sensors for recording agronomically relevant parameters, as well as many farm management systems.Electronically controlled machines and robots are state of the art.In fact, technology is now capable of automating cyber-physical systems by networking between different machines.This is what we call "agriculture 4.0".However, it still cannot be claimed that precision agriculture has been widely established in crop production.Why not?Data alone are not enough.Automatic data recording only helps farm results where the analysis of the collected material takes less time and allows more profit to be made compared with good management decision based on gut feeling and experience.Today, the largest portion of added value deriving from the new technology lies with the machinery and not the agricultural products.For instance, futures' trading on the commodity market has a much faster and more direct influence on value development of agricultural products than does, e.g., the quality of the product, or its yield, being increased in single-figure percentages through the application of site-specific management techniques.
There remains the advantage of time savings.Hereby, the task of agricultural robotic engineering development is the creation of intelligent and simple to operate, so-called "smart" systems.We call smart products those that appear cleverer than the user in that they deliver answers even before the question has been asked.An example: so-called fitness bracelets that record and analyze the wearer's movements.The smartness of the equipment lies in the analysis of the values.Step count and heart frequency are below average.This finding leads to a treatment recommendation: exercise more!But the user still has to carry out these recommendations.The second example, this time from precision agriculture.Easily the most successful crop plant sensors systems are those, that analyze, recommend, and then apply a treatment in one go, such as the so-called N-Sensors.Although the analytical procedure within the system is highly complex, such sensors are very easy to operate.In contrast to that, for instance yield mapping is an off-line approach which requires additional processing steps analyzing the data on the PC.On top of everything else, the yield information gained from one harvest can only be set to use in the next growing season, representing a long-term investment with much manual input and benefits that are difficult to assess.
Challenges for sensor development and agricultural robotic technology lie in the required high temporal and spatial resolution data which are very different and difficult to measure parameters under most unfavorable conditions.The aim of new analysis methods is to combine the data and to fuse the different information layers in order to derive new knowledge.Additionally, automation of the data collection tasks is a requirement in the development of "smart sensor" systems for agricultural application in the sense that the decision making is embedded in the sensor so that the results are directly applicable to the robot for carrying out precise management actions.But what exactly has the fitness bracelet mentioned above to do with the N-Sensor as far as content is concerned?Both sensors analyze data and process the material up to the point where recommendations for direct action can be deduced from the result.Additionally, both analytics are based on indicators not directly related with the actual target values.The "fitness" of the plant can be efficiently assessed through foliage chlorophyll content or green color.But where the cause of the problem is not poor nitrogen supply but instead lack of moisture, then the system must have this additional information available.
In this respect, intuitive interaction between man and robot is necessary, a point that also represents a great development challenge for sensors and automation technology in crop production.The more we appreciate the importance of the comprehension of detailed agronomic relationships, the greater the need for information towards a better understanding of these relationships.The more information available, the deeper the understanding and this requires, in turn, more data collection.The situation is therefore a loop within which, especially in recent years, more and more data was collected and increasingly intensified agronomic knowledge has been developed.Hereby, however, the practical application of directly usable agronomic knowledge has stagnated.Nowadays it still requires a considerable mass of statistics and software expertise for comprehensive application of precision agricultural technology.For further development of smart sensors, relevant information must be integrated into multi-causal decision-making systems in order to generate knowledge.The targets are complex systems that are easy to operate solutions with systemic, comprehensive and transparent concepts, with good "usability" and simple application.There must also be a way for practical experience to flow into these integrated systems so that farmers, with the help of the technology, can develop their expertise further.A core theme in the development of Ag-robotics decision support systems is the step from data storage over information retrieval to knowledge management involving large amounts of data.
Currently, possibilities for the analysis of agricultural data and sensor data fusion are being expanded through the application of multivariate statistical methods and machine learning techniques.Hereby, the system limits are increasingly expanded and nowadays holistic concepts for complete added-value networks are already in focus, whereby the mobile transmission of data is a basis technology for the establishment of fully integrated systems enabling real-time data fusion from different sources.
What we have discussed here are knowledge management and intelligent systems.But, with all this "high-tech", are we able to concentrate clearly on our target?The crux of all the technical developments is creating a more efficient crop production.Automation and networking should serve the systemic control of the agronomic processes, not vice versa.This is the environment in which the Leibniz Research Alliances' "Sustainable Food Production and Healthy Nutrition" innovation initiative "Food and Agriculture 4.0" focuses on the agricultural production processintelligently connected, of course.The aim of the initiative is the interdisciplinary development of process technology basics for Agriculture 4.0.Where knowledge-based decision making shall ensure the satisfaction of social demands as well as individual producer's and consumer's requirements, in terms of yields and profits, while still taking into account the local, spatial, environmental heterogeneities as well as global climate phenomena.For this purpose the research goal is to develop on one hand models of the agricultural production processes, adjusted to meet the specific conditions, and on the other hand automation technologies with which the processes shall be controlled such that the natural resources can be retained or even improved and at the same time the product quality will be maintained.
The interoperability and digital networking of agriculture will enable new process control systems and new sales models such as online slurry sales points, exchange platforms where data is traded for advice or online direct marketing.However, even with Agriculture 4.0, only what is sown can be driven home from the field.For instance, the weather risk will not be any the less, although the harvesting window might be better positioned by setting information technology to use.Hereby we finish with the summary that even Agriculture 4.0 will show only modest results if we do not take care that some of the value added through the new technologies is actually being associated with the agricultural products.

Conclusions
Research efforts for development of agricultural robots that can effectively perform tedious field tasks have grown significantly in the past decade.With the exception of milking robots that were invented in the Netherlands, robotics has not reached a commercial scale for agricultural applications.With the decrease of the workforce and the increase of production cost, research areas on robotic weeding and harvesting have received more and more attention in the recent years, however the fastest available prototype robots for weeding and harvesting are not even close to being able to compete with the human operator.For the case of picking valuable fruits using robots, the technology is now becoming closer to a commercial product with the emerging of the SWEEPER.For other fruits such as citrus and apples that can be mass harvested for juice industry, modifications of the existing mechanical harvesting systems with some robot functionalities may be more promising than using single robot system.Increasing the speed and accuracy of robots for farming applications are the main issues to be addressed for generalization of robotics systems, however, compared to the industrial and military cases, the lack of abundant research funding and budgets in agriculture has decelerated this process.For the case of robot harvesting, improving sensing (fruit detection), acting (manipulator movement, fruit attachment, detaching, and collecting), and growing system (leave pruning and plant reshaping) are suggested to increase the efficiency.It should be noted that development of an affordable and effective agriculture robot requires a multidisciplinary collaboration in several areas such as horticultural engineering, computer science, mechatronics, dynamic control, deep learning and intelligent systems, sensors and instrumentation, software design, system integration, and crop management.We highlighted some of the facing challenges in the context of utilizing sensors and robotics for precision agriculture and digital farming as: object identification, task planning algorithms, digitalization, and optimization of sensors.It was also mentioned that for an autonomous framework to successfully execute farming tasks, research focus should be toward developing simple manipulators and multi-robot systems.This is in fact one of the academic trends and research focuses in agricultural robotics for building a swarm of small-scale robots and drones that collaborate together to optimize farming inputs and reveal denied or concealed information.As of the conclusion, some forms of human-robot collaboration as well as modification of the crop breeding and planting systems in fields and greenhouses might be necessary to solve the challenges of agricultural robotics that cannot yet be automated.For example, in a collaborative harvesting system using human-and-robot, any fruit that is missed by the robot vision will be spotted by the human on a touchscreen interface.Alternatively, the entire robot sensing and acting mechanism can be performed by a human operator in a virtual environment.Nevertheless, an agricultural robot must be economically viable which means it must sense fast, calculate fast, and act fast to respond to the variability of the environment.

Figure 1
Figure 1 SWEEPER robot in action: The world first fully automated sweet pepper harvesting platform

Figure 2
Figure 2 Example of weed control and targeted spraying robots

Figure 5
Figure 5 Examples of harvesting robots for different fruits

Figure 7
Figure 7 Examples of Multi-Robot systems for digital farming