This workpackage aims to build a demonstrator for sweet pepper production. The focus of the research is on robotic harvesting of sweet pepper fruit.
Research will also emphasize development of generic concepts that can be used to perform other functions such as planting, attaching plant to supporting wires, pruning, leaf picking, monitoring yield and quality and local crop protection.
Final demonstrator for harvesting sweet-pepper with all modules integrated
The developed design is based on the current good practice cropping system of sweet-peppers in the Netherlands. The base of the robot consists out of a carrier platform on which a robotic arm with the end-effector to grip and cut the fruit, a compressor for the pneumatics, the control electronics, the computers and the sensors for fruit and obstacle detection are mounted.
In April 2014 it was proven in a commercial greenhouse that the system can harvest ripe fruits fully autonomously. In the upcoming weeks the performance of the system in terms of harvest success, cycle times and causes of failures will be determined. The following screenshots are from a video clip that shows a harvest cycle in the greenhouse.
The speed of the robot during this first harvest operations in real-world conditions is slowed down by purpose to prevent damages on plant and robot due to unforeseen issues. The video can be downloaded here (File size=16.6 MB), more video clips are available in the videos section of the website.
During the project different concepts for fruit and obstacle detection, ripeness determination and localisation and concepts to reach, grasp and detach fruit were investigated. In the final demonstrator localisation of ripe fruit takes place at two different stages. First, a side view image of the canopy is taken by a colour camera and a Time of Flight camera. The images of both cameras get registered to have full information on colour combined with 3d data. After positioning the arm in front of the target fruit a colour camera and a 3d camera integrated in the robotic hand are used to recalculate the fruit position with high accuracy.
Example image with localised targets from overview cameras (left) and camera on the gripper (right)
Obstacles not only block access to the fruit and also reduce visibility. The integrated obstacle detection is currently limited to detect and map hard obstacles such as the plant stems and non-target fruits. The method applied makes use of a set of small baseline stereo images acquired by the two overview color cameras on the main platform.
Obstacle (main plant stems) detection result and visualisation of the obstacles in the RVIZ simulation environment
Two different end-effectors to detach fruits from the plant were developed and analysed: a gripper with an integrated cutting tool with fingers that are based on the Finray principle and a guidance and cutting tool approaching the fruit from below. On both types of end-effectors two mini cameras are mounted: one Time of Flight camera and one colour camera.
Gripper with an integrated cutting tool with fingers that are based on the Finray principle
The software of the robot is implemented for the Linux operating system and the middleware ROS. The main software is based on a finite state machine and includes diagnostic tools and performance measures of the harvesting operation.
Concepts for a number of cultivation related tasks for sweet-pepper production other than harvesting were analysed. Four tasks were identified to be promising to automate which are: Monitoring, planting, pruning and tying up the plant. A mobile platform that can move between the plant rows, as under development in this project offers opportunities also for plant monitoring. The other tasks are most likely not economically feasible to automate.
In co-operation with WP13 the economic feasibility for sweet pepper harvesting was calculated. A comparison was made between manual harvesting and fully robotized harvesting, with the return on investment or investment space as a result. With a cycle time of 6 seconds for each harvested product, the sweet pepper robot will be profitable when the price of one robot will be below € 195.000 at an economic and technical life cycle of 5 years. One robot can cover 1.4 Ha of a modern Dutch sweet pepper Greenhouse.
April 2012: first prototype of a manipulator tested for the first time in a sweet-pepper crop
Main partners involved in workpackage 5
- Jentjens Machinetechniek b.v.
- A growers advisory board with several Dutch sweet-pepper growers
Lead partner of this workpackes is Wageningen UR.