Smart algorithm can save hundreds of hours of work when processing images

Published on
April 19, 2022

More and more robots are being used in agriculture and horticulture. The performance of these robots is largely dependent on algorithms that can process images of crops. For the algorithms to work properly, humans have to select and label images, which is a time-consuming task. A new active learning algorithm, developed by PhD student Pieter Blok, is able to determine which images are the most informative. This innovation can provide enormous labour savings in the world of agri-food robotics.

Through deep learning, robotic systems in the agricultural sector are getting better at recognising and processing photos of crops. To properly train the systems, hundreds to thousands of images are needed of the crop for which they are used. These images are largely processed by humans. First they find out which photos are relevant and then they have to annotate them: draw in pixels and label them. Based on this information, a system learns to recognise patterns that they will encounter in the greenhouse or on the field.

Selecting and annotating images by experts is an expensive, labour-intensive job that can involve human error. Pieter Blok, PhD student at Wageningen University & Research, went in search of a method that could simplify the selection and annotation of images. He designed an algorithm that uses active learning, a smart sampling technique. The algorithm is able to select the most interesting images from a dataset itself. Those are the images the system has trouble with, and can therefore learn the most from.

Trial with broccoli

To test his algorithm, Blok set to work with photos of broccoli plants in the field. The image below demonstrates how. Three different attempts by the system to process the broccoli produced three different outputs on the same image: sick (left), overripe (center), and healthy (right). So the uncertainty factor was very large, and the algorithm concluded that this was an interesting image for humans to label. The tagged image was then fed back to the system so that it learns from it. Blok: 'If the algorithm is uncertain about something, there is probably a lot of room there to improve its performance.'

By leaving only the 'uncertain' images to humans, Blok's method saved a lot of time. With random sampling, people would have had to label 2,300 images, but because of his smart sampling, that number was reduced to only 900 images. So a saving of 1,400 images. Blok does a quick calculation: 'Annotating one image takes about 3 to 5 minutes. If you save 1,400 images, like in my dataset, you're talking about a time saving of maybe 7,000 minutes in total. That is over 116 hours.”

Freely accessible

According to Blok, all companies in the agricultural sector are confronted with the fact that annotating images takes a lot of time. He therefore hopes that his innovative method will be widely used. His software, which has been given the name MaskAL, is free to download.

Pieter Blok conducted his research on behalf of the top sector TKI AgroFood. In addition, the companies Agrifac Machinery B.V. and Exxact Robotics made a financial and technical contribution.