Photo caption: Members of the University of Canterbury's UC Vision team, from left: Matt Mattar, Dr Richie Ellingham, Dr Oliver Batchelor and Professor Richard Green
Te Whare Wānanga o Waitaha | University of Canterbury (UC) Computer Science Professor Richard Green leads UC Vision with researcher Dr Richie Ellingham developing AI intelligence and computer vision technology that can identify, measure and track fruit hidden behind dense foliage.
"You can't automate what you can't see. By creating a complete 3D model of each plant, including what is behind the leaves, we can open the door to automating almost any repetitive manual task associated with that plant," Professor Green says.
He says robots have been used on factory production lines for decades because they work from precise computer-aided design models.
"Agricultural environments are far more complex because every tree, vine, branch and piece of fruit is different."
The UC Vision team's technology addresses that challenge by using cameras, lighting and a complex pipeline of AI systems to digitally remove foliage and reconstruct an accurate 3D model of a plant.
Professor Green says the technology is the first of its kind to create a complete 3D model of individual plants, including fruit and structures hidden behind foliage, to support crop forecasting and future automation.
"The system can identify individual apples, cherries or grapes, show which branch or cane they are attached to, and calculate measurements including their size, volume and surface area. Repeated scans can also track how much an individual piece of fruit has grown."
Professor Green says this information could significantly improve crop forecasting.
"Growers currently rely largely on people to sample, count and measure fruit before harvest, a process that can produce inaccuracies of up to 23%."
The UC technology has achieved fruit counts within about 2% to 3% of the correct figure during testing.
"The result reflects the combined expertise of the research team, including UC researcher Dr Oliver Batchelor on AI algorithms, Dr Ellingham's mechatronics research and UC research engineer Matt Mattar's development of commercial-quality software alongside specialist research engineers selected for key areas of the project," Professor Green says.
"For large growers, better information could potentially save millions of dollars and reduce waste across their operations.
"Every vineyard and orchard needs to understand how much fruit it will produce and the size distribution of that fruit. Getting that wrong can mean having too many or too few workers, packaging materials or storage resources available at harvest time."
UC Vision has developed mobile camera systems that travel along vineyard and orchard rows. The vineyard system uses two rows of cameras to capture images through dense vine canopies, while a taller camera structure can scan orchard trees reaching approximately 3.5 metres.
Professor Green says the detailed models could eventually guide autonomous machines to carry out tasks such as pruning, thinning, spraying, and harvesting.
"In a cherry orchard, for example, a machine could identify and pick only fruit that had reached the most valuable size, returning later for the remaining crop.
"Cherries that are only a few millimetres larger can be worth twice as much. Selective harvesting could help growers increase the value of their crop while also planning their workforce and packaging requirements more accurately."
The breakthrough builds on about 15 years of UC research projects, supported by more than $32 million in government investment in computer vision, artificial intelligence and agricultural robotics.
The team is now concentrating on making the system highly reliable ahead of potential commercialisation through HoloCrop.
Dr Ellingham says the team has collected 3D modelling data across 20 commercial farms to ensure the technology works reliably in real vineyard and orchard conditions.
"Almost every farm we work with is asking when they can start using our sampling tools," Dr Ellingham says.
"HoloCrop is being established to meet demand across the fruit production chain by developing precision horticulture tools that provide more accurate data, reduce food-production waste and support future horticultural robotics."
Professor Green says the technology is the result of years of learning, testing and refinement by an exceptionally talented team.
"We now have technology with the potential to change how food is grown and harvested around the world."
