Harvey – The Robotic Capsicum Harvester

Why it matters

In Australia, fruit and vegetable farmers face high labour costs ranging from 20% to 40% of total operational costs, reducing the farmer’s ability to operate a profitable business. In addition, farmers often experience a shortage in skilled labour, especially during optimal harvesting periods. In some cases, the availability and cost of employing pickers to harvest crop proves to be nonviable, resulting in valuable crops remaining unharvested.

Despite significant efforts by the worldwide horticultural research industry, progress in creating robots to harvest capsicums has been modest to date.

Project overview

QUT has developed a prototype robotic capsicum (sweet pepper) harvester nicknamed ‘Harvey’, combining robotic vision and automation expertise to benefit agricultural producers.  The project, which ran from 2013 – 2017,  was led by QUT researchers and engineers with significant co-funding from the Queensland Government Department of Agriculture and Fisheries.

When developing a harvesting robot, the primary barrier roboticists face globally is the perception of key parts of the crop and the effectiveness of the actual harvesting tool. In early 2017, worldwide literature indicated a success rate of only 6% in testing scenarios similar to those used for Harvey, and up to 30% when the crop is modified and leaves are removed.

During testing at multiple protected cropping facilities across Queensland, Harvey demonstrated outstanding progress due to its sophisticated robotic-vision algorithms and novel end-effector tool design.  Data from a camera-in-hand system moves with the robot—rather than being fixed— to create a 3D representation of the fruit and its surroundings, this is processed to extract key information for the execution of the harvesting operation. This information is used by a robot motion planning algorithm that commands the action of a novel multi-modal tool for detaching the fruit. The camera system and the harvesting tool are mounted at the end of a standard robotic (arm) manipulator. The combination of robotic-vision techniques and crop manipulation tools are key enabling factors for the harvesting of these crops.

Real world impact

The team are planning to extend the technology and research developed for Harvey beyond the current project to deal with other crops such as mangoes, strawberries, tomatoes, apples and avocados.  We believe the next agricultural revolution will be driven by agri-intelligent systems. The development and application of technology such as Harvey will help farming and food enterprises operate more efficiently, profitably and sustainably.

Harvey was recently tested at a protected cropping facility in Townsville and Brisbane (Redlands), Queensland.  The most recent results show:

• a fruit harvesting success rate of 76.5% in modified crop scenarios (some leaves removed or moved pre-harvesting)
• an average harvesting time of approximately 30 seconds per fruit.

There is still room for improvement given that Harvey is an initial proof of concept. Based on the observed robot behaviours in the trials, the team has already identified areas of improvements and estimates that an overall success rate in excess of 90% will soon be achievable with only minor modifications.

Papers:

1. Lehnert, C., McCool, C., Sa, I., & Perez, T. (2020). Performance improvements of a sweet pepper harvesting robot in protected cropping environments. Journal of Field Robotics. https://doi.org/10.1002/rob.21973
2. Lehnert, C., McCool, C., Sa, I., Perez, T., English, A., McCool, C., … Perez, T. (2017). Autonomous Sweet Pepper Harvesting for Protected Cropping Systems. IEEE Robotics and Automation Letters, 2(2), 872–879. https://doi.org/10.1109/LRA.2017.2655622