In the Dynamic Machine Learning (DML) project, which started in 2021, researchers are working on machine-learning techniques that can reuse previously acquired knowledge and transfer it to each new cultivar within a crop. One of the work packages aims to further develop an autonomous robot that can predict the yield capacity of a tomato greenhouse: the Plantalyzer. The goal is self-learning detection for all tomato varieties.
In the Netherlands, tomatoes are grown on just under 1900 ha. With 49%, the vine tomato takes up the largest cultivation area. Wageningen researchers from Agro Food Robotics are now developing a technique for the reliable detection of vine tomatoes in the greenhouse using an autonomous robot.
Accurate harvest forecasting
Growers want to be able to estimate how many tomatoes can be harvested in the coming period as accurately as possible to optimally meet market demand. By controlling temperature, water and light, growers can influence the ripening of the tomatoes. This requires a lot of experience, and in these times of labor scarcity in horticulture, the sector could use an efficiency boost. Harvest forecasting with an autonomous robot such as that of project partner Berg Hortimotive – Hortikey could offer a solution in the near future.
The autonomous robot that can predict the yield capacity of a tomato greenhouse: the Plantalyzer
Increased tomato detection thanks to stereo cameras and 3D clustering
Franck Golbach, work package leader and researcher at Wageningen University & Research Agro Food Robotics: 'Last year, we started using machine learning and deep learning techniques for the detection of tomatoes. This year we completely revamped the way we detect tomatoes and trusses. The detection rate, the reliability with which we can see and locate vine tomatoes in the greenhouse, has been greatly increased as a result.'
The team uses stereo cameras to determine the 3D position, or real-world position, of vine tomatoes in the greenhouse. The trusses are found using a 3D clustering algorithm. The color, and thus the ripeness of tomatoes, is measured using a color model that takes into account different lighting conditions.
Detecting the ideal ripeness in vine tomatoes
There are tomato varieties that are picked per tomato, but vine tomatoes are, as the name suggests, cut per truss on the vine. A truss begins to ripen at the top and is harvested before the entire truss is red and ripe. After cutting, the ripening continues so that the consumer eventually buys a nice, evenly ripe truss. Franck: "It is crucial that we can discover and recognize that ideal maturity stage of the truss so that growers can harvest at exactly the right time. What makes it even more challenging: we don't just want to look at the truss but also at the individual tomatoes in the truss. This is difficult because some fruits are partly or completely behind each other or are covered by leaves. That is why the robot takes as many photos as possible, and we combine all that information. We are currently taking a large number of photographs to analyze and verify the consistency and accuracy of our algorithms."
Learning from data
Several Plantalyzers are already working in greenhouses to collect data. At one grower, the team is collecting data from the entire season. After this data has been processed in the Plantalyzer, it will be used by project partner LetsGrow to further improve the ripening model for growers. This will allow growers to better analyze the influence of various growth parameters on the yield. The dataset will also be used to fine-tune the current deep-learning-based models to deal with changes over the season.
"If we can indeed demonstrate that we can find the trusses well and reliably assess ripeness based on the dataset, we will have reached a milestone that we can be satisfied with!"
One machine for all tomato varieties
If the project team succeeds in the 3-4 tomato varieties, they are currently assessing, that offers great perspective for the future. It would mean that an autonomous robot like Plantalyzer is suitable for all tomato varieties and make it possible for growers to store and exchange knowledge. The intention is that the machine also becomes self-learning: if there is a new tomato variety, it must be able to store and share that new knowledge. That way, Plantalyzer can keep developing.
"Many good things have already happened, but this is a very challenging project, so we continue working in good spirits," Franck concludes.
For more information:
Wageningen University & Research
www.wur.nl